Skip to main content

Posts

Showing posts from November, 2017

Apache spark word count example with scala

package com.ercancan.spark import org.apache.log4j.{Level, Logger} import org.apache.spark.{SparkConf, SparkContext} /** * Spark word count example * @author Ercan Can */ object WordCount { //Main function which implements the primary logic of SPARC job def main(args: Array[String]) { /* Windows binaries for Hadoop versions Apache Spark requires the executable file winutils.exe to function correctly on the Windows Operating System when running against a non-Windows cluster. */ System.setProperty("hadoop.home.dir", "C:\\winutils") Logger.getLogger("org").setLevel(Level.ERROR) /* Create the SPARK configuration. This is entry point to SPARK applicaion. Our application named as "wordcount". We're going to run our application on an embedded SPARK instance on our local box which cold use up to two core cpu. */ val conf = new SparkConf().setAppName("wor...

PHP Wordpres installation on EC2 Ubuntu Instance

--  Launch an EC2 Ubuntu Instance from below link http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/EC2_GetStarted.html -- Create new certificate and after download give permisson to pem file chmod 400 mycertificate.pem -- Connect to EC2 instance ssh -i mycertificate.pem ubuntu@eec2-**-**-**-**.eu-west-1.compute.amazonaws.com -- Update ubuntu sudo apt-get update -- Installing mysql server $ sudo apt-get install mysql-server root pass : ************ $ sudo mysql_secure_installation     Enter current password for root: Press return for none     Change Root Password: Y     New Password: Enter your new password     Remove anonymous user: Y     Disallow root login remotely: Y     Remove test database and access to it: Y     Reload privilege tables now: Y $ sudo vim /etc/mysql/mysql.conf.d/myconf.cnf     [mysqld]     lower_case_table_names=1   ...

Redis spring boot code snipets

# application properties spring.redis.database=0 spring.redis.host=192.168.99.100 spring.redis.port=6379 spring.redis.password=******* spring.redis.pool.max-active=8 spring.redis.pool.max-wait=-1 spring.redis.pool.max-idle=8 spring.redis.pool.min-idle=0 spring.redis.timeout=0 # Bean definition in configuration class, for manual access redis server @Bean @ConditionalOnMissingBean(name = "redisTemplate") public RedisTemplate<Object, Object> redisTemplate(RedisConnectionFactory redisConnectionFactory) throws UnknownHostException {     RedisTemplate<Object, Object> template = new RedisTemplate<Object, Object>();     template.setConnectionFactory(redisConnectionFactory);     return template; } # In service classes inject redis templates @Autowired private RedisTemplate<String, Object> redisTemplate; @Autowired private StringRedisTemplate stringRedisTemplate; public void testRedis(){   ...

Redis client operations with python on Jupyter

In [1]: import redis conn = redis . Redis ( host = '192.168.99.100' , port = 6379 , password = '*******' ) In [2]: conn . set ( 'key1' , 'val1' ) value = conn . get ( 'key1' ) print ( value ) val1 In [3]: import json user = { "id" : 1 , "name" : "Ercan" , "surname" : "Can" } userd = json . dumps ( user ) userl = json . loads ( userd ) print userl [ 'id' ], userl [ 'name' ] 1 Ercan In [4]: import json ercan = { "id" : 1 , "name" : "Ercan" , "surname" : "Can" } json . dumps ( ercan ) conn . set ( 'user_1' , json . dumps ( ercan )) json . loads ( conn . get ( 'user_1' ))[ 'id' ] 1 In [5]: print conn . keys () conn . delete ( 'key2' ) x = conn . get ( 'use...

Solution to Docker certification error on Windows Server

"certificate signed by unknown authority error." Get the set of corporate root certificates, which should be installed in your corporate-configured browser. Export each (I have two), one at a time, making sure to choose DER format. 1) Create this directory c:\Users\username\certs and copy cer file here. 2) Open Docker Quickstart $ cd /c/Users/username/certs 3) Convert a DER file (.crt .cer .der) to PEM $ openssl x509 -inform der -in certificate.cer -out certificate.pem 4) Connect to default machine $ docker-machine ssh default docker@default:~$ sudo -s root@default:/home/docker# mkdir /var/lib/boot2docker/certs root@default:/home/docker# cp /c/Users/my.username/certs/*.pem /var/lib/boot2docker/certs/ touch /var/lib/boot2docker/bootlocal.sh && chmod +x /var/lib/boot2docker/bootlocal.sh vi /var/lib/boot2docker/bootlocal.sh Insert the following and save the file: #!/bin/sh mkdir -p /etc/docker/certs.d && cp certs/certificate.pem /etc/docke...