Latest Event Updates

How to Reset MySQL >= 5.7.6 Password

Posted on Updated on

  1. Login to the MySQL Server Host.
  2. # service mysql stop
  3. Start MySQL with skip grant table
    # mysqld_safe –skip-grant-tables &
  4. Login to MySQL as root (without password)
    # mysql -u root
  5. Let say we want to reset root password to ‘jangkrik‘, then we do:
    mysql> flush privileges;
    mysql> ALTER USER ‘root’@’localhost’ IDENTIFIED BY ‘jangkrik‘;
    mysql> flush privileges;
    mysql> quit;
  6. Check if the root password is set correctly
    # mysql -u root -p
  7. If all working as intended, Then shutdown MySQL
    # mysqladmin shutdown -p
  8. Start MySQL daemon normally
    # service mysql start

How to Render a Html Page with Selenium Webdriver + PhantomJS in Python

Posted on Updated on

For a better view, check out the Github link.

#
# 20160929 - by sphinxid - firman.gautama@gmail.com
#
# Example of multithreaded selenium webdriver with phantomjs in Python.
# In this example, it will use 10 thread + 10 phantomjs to do 25000 request to "url".
# 

from selenium import webdriver
import time
import concurrent.futures
import signal
from concurrent.futures import ThreadPoolExecutor
from random import randint

def fetch(url, driver):
 try:
 if not driver.current_url:
 driver.refresh()
 else:
 driver.get(url)

 driver.implicitly_wait(2)
 driver.set_page_load_timeout(2)
 print 1
 except:
 print 2
 pass

 return 0

def clean_up(driver):
 try:
 driver.service.process.send_signal(signal.SIGTERM)
 driver.quit()
 except:
 pass

 return

if __name__ == "__main__":
 num_thread = 10
 num_request = 25000
 url = "http://www.yahoo.com/"

 # instantiate threadpool
 pool = ThreadPoolExecutor(num_thread)
 parr = []

 # instantiate PhantomJS per THread
 for x in range(0, num_thread):
 print "Initialized thread %s " % x
 parr.append(webdriver.PhantomJS())
 print " OK."

 start_time = time.time()

 # Use one random thread from thread pool to access the URL
 for x in range(0, num_request-1):
 n = randint(0, (num_thread-1))
 future = pool.submit(fetch, url, parr[n])
 future.done()

 # clean_up: make sure phantomjs process is closed
 for x in range(0, num_thread):
 future = pool.submit(clean_up, parr[x])
 future.done()

How to Convert Compressed Text: bzip2 to gzip without Temporary Files Multithreaded

Posted on Updated on

The tools that you will need is: lbzip2, pigz and split.

lbzip2 => http://lbzip2.org/
pigz => http://www.zlib.net/pigz/

If you are using Ubuntu (I’m using 14.04 LTS).
You can easily install lbzip2 and pigz by using apt-get (aptitude).

$ apt-get install lbzip2 pigz

Or you could download the source code from their website and compile manually.

Let say you have 500gb of text compressed bzip2 files called file01.txt.bz2 and you want to split that file to a multiple gziped files with 1500000 lines each, so it will be able to be processed faster in your hadoop cluster.

$ lbzcat file01.txt.bz2 | split -d -a 10 -l1500000 --filter='pigz > newfile01-$FILE.gz'

 

How to Export/Import HBase Table

Posted on Updated on

– EXPORT –

eg, hbase table: hbase_test_table
and today date is 20160820

1. Create a temporary folder in hdfs for the exported files:

$ hadoop fs -mkdir /tmp/hbasedump/20160820

2. Execute this shell command in any hadoop node that has hbase gateway

$ hbase org.apache.hadoop.hbase.mapreduce.Export hbase_test_table /tmp/hbasedump/20160820/hbase_test_table

3. Please don’t forget to get the table structure, so you will be able to import the data back later on if needed.

$ hbase shell
 hbase-shell> describe 'hbase_test_table'
Table hbase_test_table is ENABLED
 hbase_test_table
 COLUMN FAMILIES DESCRIPTION
 {NAME => 'test_cf', DATA_BLOCK_ENCODING => 'NONE', BLOOMFILTER => 'NONE', REPLICATION_SCOPE => '0', COMPRESSION => 'SNAPPY', VERSIONS => '1', MIN_VERSIONS => '0', TTL => 'FOREVER'
 , KEEP_DELETED_CELLS => 'FALSE', BLOCKSIZE => '65536', IN_MEMORY => 'false', BLOCKCACHE => 'false'}
 1 row(s) in 0.1290 seconds

– IMPORT –

eg, hbase table: test_import_hbase_test_table

1. Let say you have the dumped export file for that table in (hdfs) /tmp/hbasedump/20160820/hbase_test_table
And you want to import it to a new table “test_import_hbase_test_table”

2.

$ hbase shell

– Create the table if it’s not yet created “test_import_hbase_test_table”
– Create the table with the same column family name (get the information on the export step #3 above).

3. Start the import process:

$ hbase org.apache.hadoop.hbase.mapreduce.Import "test_import_hbase_test_table" "/tmp/hbasedump/20160820/hbase_test_table"