Skip to main content

Posts

Showing posts from August, 2012

Mongo cluster configuration

sudo mkdir -p /data/db sudo chmod 0755 /data/db sudo chown -R ercan /data/ sudo mongod --replSet ClsName --logappend --logpath /data/log --pidfilepath ./data/mongodb.pid --fork --dbpath /data/db --directoryperdb --bind_ip serverIP1 --port 27017 sudo mongod --replSet ClsName --logappend --logpath /data/log --pidfilepath ./data/mongodb.pid --fork --dbpath /data/db --directoryperdb --bind_ip serverIP2 --port 27017 sudo mongod --replSet ClsName --logappend --logpath /data/log --pidfilepath ./data/mongodb.pid --fork --dbpath /data/db --directoryperdb --bind_ip serverIP3 --port 27017 mongo serverIP1:27017 ClsName:PRIMARY>rs.status() ClsName:PRIMARY>rs.initiate() ClsName:PRIMARY>rs.isMaster() ClsName:PRIMARY>rs.add("serverIP2:27017") ClsName:PRIMARY>rs.conf() ClsName:PRIMARY>rs.add("serverIP3:27017") ClsName:PRIMARY>rs.conf() ClsName:PRIMARY>rs.status() mongo serverIP2:27017 ClsName:SECONDARY>db.getMongo().setSlaveOk() m...

Hadoop

Hadoop is a free, Java-based programming framework that supports the processing of large data sets in a distributed computing environment. Hadoop Offers:  Hadoop Common: The common utilities that support the other Hadoop subprojects. HDFS: A distributed file system that provides high throughput access to application data. MapReduce: A software framework for distributed processing of large data sets on compute clusters. Avro: A data serialization system. Chukwa: A data collection system for managing large distributed systems. HBase: A scalable, distributed database that supports structured data storage for large tables. Hive: A data warehouse infrastructure that provides data summarization and ad hoc querying. Mahout: A Scalable machine learning and data mining library. Pig: A high-level data-flow language and execution framework for parallel computation. ZooKeeper: A high-performance coordination service for distributed applications. Oozie – Workflow for interdependent ...

mysql backup/restore

mysqldump -u dbUser -p dbPassword YOUR_SCHEMA | gzip -9 > YOUR_SCHEMA.sql.gz mysql -u dbUser -p dbPassword < gzip -cd YOUR_SCHEMA.sql.gz mysqldump --default-character-set=utf8 --user=dbUser --password=dbPassword YOUR_SCHEMA | gzip -f > db.sql.gz mysqldump -hlocalhost -udbUser -pdbPassword --skip-opt --add-drop-table YOUR_SCHEMA > YOUR_SCHEMA.sql mysqldump -hlocalhost -udbUser -pdbPassword --skip-opt --order-byprimary --skip-dump-date --skip-comments --add-drop-table YOUR_SCHEMA > YOUR_SCHEMA.sql mysqldump -hlocalhost -udbUser -pdbPassword --skip-add-drop-table --skip-add-locks --skip-disable-keys --skip-set-charset YOUR_SCHEMA > YOUR_SCHEMA.sql mysqldump -hlocalhost -udbUser -pdbPassword --compact --skip-comments YOUR_SCHEMA > YOUR_SCHEMA.sql mysqldump --user=dbUser --password=dbPassword --host=localhost --port=3306 --extended-insert=FALSE --add-drop-database=TRUE --add-drop-table=TRUE --disable-keys=TRUE --order-by-primary=TRUE --routines=TRUE --trigge...

JSON document insert in mongoDB

Following JSON "document" can be stored in Mongo DB: { name: 'ercan',   articleCreateDate : new Date('03/28/2011'),   title : 'MongoDB insert query sample...',   detail : { columnCount : 11, columnSize : "not set", active : "true"   },   tags : [ 'mongo', 'insert' ],   comments : [ { name: 'ercan', comment: 'I disagree' },                { name: 'can', comment: 'Good post' }            ] } This document stored in a "articles" collection using the shell: > doc = { name: 'ercan', ...   articleCreateDate : new Date('03/28/2011'), ...   title : 'MongoDB insert query sample...', ...   detail : { ... columnCount : 11, ... columnSize : "not set", ... active : "true" ...    }, ...   tags : [ 'mongo', 'insert' ], ...   comments : [ { name: 'ercan', comment: 'I disagree...

CSV file import in MongoDB

users.csv like following;    ercan,can,engineer    can,ercan,author $sudo mongoimport -v -d test -c user -f firstName,lastName,job --type csv --file users.csv (after execute this command, the following output occured) Tue Aug 14 14:02:45 creating new connection to:127.0.0.1 Tue Aug 14 14:02:45 BackgroundJob starting: ConnectBG Tue Aug 14 14:02:45 connected connection! connected to: 127.0.0.1 Tue Aug 14 14:02:45 ns: test.user Tue Aug 14 14:02:45 filesize: 31 Tue Aug 14 14:02:45 got line:ercan,can,eng Tue Aug 14 14:02:45 got line:can,ercan,author Tue Aug 14 14:02:45 got line: Tue Aug 14 14:02:45 got line: imported 2 objects

Connecting to Mongo Repplication Server from client with java driver

  List serverAddresses = new ArrayList();      serverAddresses.add( new ServerAddress( "serverIP1" , 27017 ) );      serverAddresses.add( new ServerAddress( "serverIP2" , 27018 ) );      serverAddresses.add( new ServerAddress( "serverIP3" , 27019 ) );  Mongo mongo = new Mongo( serverAddresses ); By default, all read and write operations will be made on the master. But it's possible to read from the slave(s) by using slaveOk. mongo.slaveOk(); 

MongoDB Connection Pool adjusted with other parameter in MongoOptions

Mongo database connection with its own internal pooling.D efault pool has a maximum of 10 connections per mongodb host. T his should be use class as a singleton. This value is controlled by the variable   connectionsPerHost within the class com.mongodb.MongoOptions . To adjust the MongoOptions you simply create an instance of MongoOptions and directly set the variables as required. MongoOptions mongoOptions = new MongoOptions (); mongoOptions . connectionsPerHost = 20 ; mongoOptions . threadsAllowedToBlockForConnectionMultiplier = 100 ; mongoOptions . maxWaitTime = 1000 ; mongoOptions . connectTimeout = 3000 ; mongoOptions . socketTimeout = 3000 ; mongo = new Mongo ( parseServerAddresses ( "localhost:27017" ), mongoOptions ); Check the number of connections in mongo db shell. >db.serverStatus()

Installation MongoDB on Ubuntu

## Authenticity by requiring that distributors sign packages with GPG keys. sudo apt-key adv --keyserver keyserver.ubuntu.com --recv 7F0CEB10 cd /etc/apt/sources.list.d sudo touch 10gen.list sudo nano 10gen.list ## add in document this command deb http://downloads-distro.mongodb.org/repo/debian-sysvinit dist 10gen sudo apt-get update ## Now issue the following command to reload your repository: sudo apt-get install mongodb-10gen ## Issue the following command to install the latest stable version of MongoDB: sudo apt-get install mongodb-10gen ## Successfully installed MongoDB ## You can connect to your MongoDB  with mongo command mongo ## This will connect to the database running on the localhost interface by default. At the mongo prompt, issue the following two commands to insert a record in the “test” collection of the (default) “test” database. > db.test.save( { a: 1 } ) > db.test.find()