tongsampah

Cloudera Enterprise VS Cloudera Express

Posted on Updated on

Cloudera Version Comparison Express Version Enterprise Version
Cluster Management
Number of hosts supported Unlimited Unlimited
Host inspector for determining CDH readiness Yes Yes
Multi-cluster management Yes Yes
Centralized view of all running commands Yes Yes
Resource management Yes Yes
Global time control for historical diagnosis Yes Yes
Cluster-wide configuration Yes Yes
Cluster-wide event management Yes Yes
Cluster-wide log search Yes Yes
Aggregate UI No Yes
Deployment
Support for CDH 4 and CDH 5 Yes Yes
Automated deployment and readiness checks Yes Yes
Installation from local repositories Yes Yes
Rolling upgrade of CDH No Yes
Service and Configuration Management
Manage Accumulo, Flume, HBase, HDFS, Hive, Hue, Impala, Isilon, Kafka, Kudu, MapReduce, Oozie, Sentry, Solr, Spark, Sqoop, YARN, and ZooKeeper services Yes Yes
Manage Key Trustee and Cloudera Navigator No Yes
Manage add-on services Yes Yes
Rolling restart of services No Yes
High availability (HA) support Yes Yes
CDH 4 – HDFS and MapReduce JobTracker (CDH 4.2) Yes Yes
CDH 5 – HDFS, Hive Metastore, Hue, Impala Llama ApplicationMaster, MapReduce JobTracker, Oozie, YARN ResourceManager Yes Yes
HBase co-processor support Yes Yes
Configuration audit trails Yes Yes
Client configuration management Yes Yes
Workflows (add, start, stop, restart, delete, and decommission services, hosts, and role instances) Yes Yes
Role groups Yes Yes
Host templates Yes Yes
Configuration versioning and history No Yes
Restoring a configuration using the API No Yes
Security
Kerberos authentication Yes Yes
LDAP authentication for CDH Yes Yes
LDAP authentication for Cloudera Manager No Yes
SAML authentication No Yes
Encrypted communication between Server and host Agents (TLS) Yes Yes
Sentry role-based access control Yes Yes
Password redaction Yes Yes
Data encryption with KMS No Yes
Cloudera Manager user roles No Yes
Monitoring and Diagnostics
Service, host, and activity monitoring Yes Yes
Proactive health tests Yes Yes
Health history Yes Yes
Advanced filtering and charting of metrics Yes Yes
Job monitoring for MapReduce jobs, YARN applications, and Impala queries Yes Yes
Similar activity performance for MapReduce jobs Yes Yes
Support for terminating activities Yes Yes
Alert Management
Alert by email Yes Yes
Alert by SNMP Yes Yes
User-defined triggers Yes Yes
Custom alert publish scripts Yes Yes
Advanced Management Features
Automated backup and disaster recovery No Yes
File browsing, searching, and disk quota management No Yes
HBase, MapReduce, Impala, and YARN usage reports No Yes
Support integration No Yes
Operational reports No Yes
Cloudera Navigator Data Management
Metadata management and augmentation No Yes
Ingest policies No Yes
Analytics No Yes
Auditing No Yes
Lineage No Yes
Advertisements

Happy Birthday to YOU

Image Posted on Updated on

happy birthday

Hack by the Beach in Jakarta

Posted on

On the past weekend, we participated in the BCA Finhacks 2016 Hackathon. It was a hackathon event at the Segarra in Jakarta Ancol beach.
There atmosphere was very very hot and humid at the day time, but they prepared for a free flow food and drinks for every participants.

Anyway.. we were creating an App called AngpaO. It’s basically an app similar to eventbrite, but 2 extra main feature which is “live streaming” of the event and a “donation” feature using BCA E-Wallet integration. If you would like to know more about the app we have our presentation slide deck uploaded here.

Our team were called “Opcode” and consisted by 3 people:
Ihsan Fauzi Rahman (Cermati.com)
Firman Gautama (ADSKOM)
Supardi (Lippo X)

What our team have done in 24 hours time frame.

1. Reverse Engineered the PHP BCA Finhacks SDK.
– Extract bca (composer phar) file into raw php.
– Implement code hook to cache ‘access_token’ from Oauth2 class from bca sdk.
— Why? Because on the briefing yesterday morning (the 1st day), we only have limit to 5 access_token request/min, and the bca sdk didn’t cache it. We were worry that during the development we could be throttled.

2. Created PHP http Wrapper for BCA API.
– Our main apps is written in NodeJS, at the present time, bca only provide sdk for php and java, so that’s why we created an internal http wrapper for our NodeJS apps.
– We implemented these following bca api on our php http wrapper:
— User Registration
— User Update
— Topup
— Payment
— History Transaction

3. Created NodeJS Frontend + API for our Web and Android App.

– We implemented these following features:
— login & register for web (+integrated with facebook account)
— user profile update
— implemented feature to create new AngpaO event.
— implemented feature to list events for other users/guest.
— implemented feature of “donation” history for event owner.
– for this, we didn’t just use bca api, because the bca api have limitation up to the last 10 transaction only. So we re-implement our own transaction history.

4. Created Android Apps (Native)

– We created a smartphone as AngpaO wallet in mind. (So every transaction should only happen via a smartphone)
– We implemented these following features:
— login & register feature from android that integrated with Facebook account
— Implemented Angpao feed events list
— Implemented Angpao event view with video live stream
— Implemented Top-up feature.
— Implemented Donate feature.
— Implemented QR code read for AngpaO event.
— Import user profile from facebook.

5. Created live streaming video+audio for our AngpaO event.
– backend: Using Wowza with RTMP & HLS protocol.
– frontend: Using jwplayer to play on the browser (we also embed this on our android apps)
– broadcaster : Using Open Broadcaster Studio and the webcam as video and audio source (for demo)

 

======
BONUS Thoughts 😀
======

1. The first version of BCA PHP SDK was broken. (On 18th April 2016 they gave everyone the SDK to download and test)
– How it broke? After we “reverse engineered the sdk” we found out that there are no getter for OAuth2Client dan JsonParser in their sdk main class loader.
(This was fixed at 23 April 2016 by BCA) (but to be prepared, we also have fixed the sdk our self, but at the 1st day of hackathon they told us if there is a new version of the sdk).
2. BCA API untuk fungsi payment-nya sempat error?
– format tanggal dibilang salah, padahal udah pake seperti yg di contoh: https://finhacks.id/api/?php#payment. (Fixed?)
–> Seems (some) of bca api server/endpoint have different timezone (GMT+5 instead of GMT+7) (how do we know this? we looked at the transaction date from bca server response)
–> We use a work around:
– Untuk payment, make sure ‘request date’ yg dikirimkan ke server < dari current datetime di server api bca, or else bakal dianggap invalid request.
– Jadinya kita sengaja mengirimkan waktu ‘request date’ yg beberapa waktu lebih lambat untuk menghindari race condition yang menyebabkan gagal nya request ‘payment’.
3. Access point wifi nya overloaded. Karena mungkin banyak-nya device peserta yang connect ke sana. Jadinya koneksi internet nya kurang stabil.
4. The bca api sandbox suddenly disappear like 12 hours before the event start 😦
5. Not every team treated equally. 
I don’t want to point out who is who, but you can ask many other people/team that participated in the event to confirm  🙂

Kenapa Uber Dipersulit Untuk Beroperasi di Indonesia Tapi Kok GoJek Ngga?

Posted on Updated on

Saat ini ada 2 startup transportasi yang sedang berkembang pesat di Indonesia, yaitu Uber dan GoJek.

Fakta nya:

  1. Mereka sama-sama penyedia startup transportasi yang beroperasi di Jakarta.
  2. Mereka sama-sama HANYA memfasilitasi pemilik kendaaran bermotor (mobil/motor) yang ingin mencari uang tambahan dengan cara mengantar penumpang dari point A ke point B.
  3. Beda nya, Uber memfasilitasi pengemudi Mobil. Sedangkan GoJek memfasilitasi pengemudi Motor.

Tapi kenapa kok Uber dipersulit di Indonesia, sedangkan GoJek tidak?

Polisi sampai menangkap dan menjebak pengemudi yang terdaftar bersama Uber karena dianggap tidak memiliki ijin beroperasi sebagai “Taksi”.

Sekarang gue tanya sama bapak2 polisi,

YANG BEROPERASI SEBAGAI TAKSI ITU SIAPA? UBER != TAKSI pak!

Terus sekarang ada GoJek, yang bisnis nya sama persis kayak Uber, tapi bedanya pake sepeda motor. Kalau Pak Polisi permasalahkan Uber menggunakan Plat Hitam, sekarang ini GoJek dan tukang Ojek lain nya juga pake plat HITAM. KENAPA KOK GAK DIPERMASALAHKAN? APALAGI KOK GAK DI DIJEBAK DAN DITANGKAP JUGA?

Ini kalau misalkan bapak polisi hanya mempersulit UBER, itu namanya diskriminasi dan tidak adil! Apakah karena Uber pake mobil dan GoJek pake motor?

Gue bukan ngebelain Uber, tapi gue merasa bapak polisi tidak adil dalam mengambil tindakan.