Hadoop has nearly become synonymous with Big Data. There is a large community backing this open source project, making it a foundation piece for several other open-source projects and tools.

In this hacknight, you will go through short tutorials on how to get started with hadoop - how to build it, get a dev environment on your laptop and write map-reduce programs in that environment.

With the help of a few convenience scripts, pre-packaged config files and some hand-holding from Gopal Vijayaraghavan, you can download hadoop, build it and set up a single node cluster without much trouble. This sets up a very simple, non-secure hadoop cluster, easily extensible to a few nodes. The multiple node setup is only useful to debug node locality and schedulers, but for most of the HDFS/Hadoop development, the single node cluster works wonders.

Once you have learnt how to set up a cluster on your system, you can dig deeper into the architecture and try to modify hadoop itself to your needs. Perhaps there’s something in there that’s annoyed you for months? Maybe you think hadoop will be better off with a DualPivotQuickSort instead of the default sort? Actually, that sounds like a good idea. And it’s not restricted to just java, perhaps you are a javascript wizard and want to fix something in the jquery based web UI?

Or maybe you’ve got a Big Data problem and hadoop looks like a good way to solve it. Maybe R on Hadoop will fit your problem? The solution might be a quick ETL phase to cut down your logs to size, where Pig might fit in neatly. Or you want to check your A/B tests and funnels, where hive does a good job. If there’s a lot of numpy you already know, perhaps Pydoop might fit the bill. And if all else fails, might as well do it in Java & call it a day.

And just in case one of those don’t work just exactly right, fixing it is always an option - after cursing, yelling and debugging, not always in that order. Or write your own.

The hacknight is open to individuals who want to work with hadoop, on hadoop and everything in the neighbourhood. Come prepared with a laptop, a github account, java skills and enough sleep to tide you over the night.

Mentors

Gopal Vijayaraghavan has spent many years making PHP faster at Yahoo! and Zynga. He thinks about performance problems as opportunities to contribute. He has turned towards Hadoop for newer challenges and is chasing performance at both ends of the scale spectrum at Hortonworks. Beyond work, he’s a photographer by day, biker by sunset, and by night he posts random thoughts.

Rohit Chattar is a Senior Architect at Yahoo! in Advertising and Data Platform group. He is a thought leader specializing in designing solutions that uses huge amounts of data. He has deep knowledge and understanding of various usage models involving traditional databases and newer Big Data platforms to provide customer centric and cost effective solutions..

What is a hacknight?

Hacknight is a space where you can work on your pet project, try a new idea or test the waters of new tools, technologies and languages. At hacknights, you meet new people, make friends and network. Overall, a hacknight is an amazing, hands-on learning experience. HasGeek's hacknights provide developers the opportunity to work in an easy, relaxed, peer-to-peer environment, over a period of 12-20 hours.

This hacknight is part of The Fifth Elephant 2013, an annual conference on big data, analytics and cloud computing. More details, visit: http://fifthelephant.in/

Next steps:

  1. Register for the hacknight. You will receive a confirmation from us.
  2. Propose an app idea, and/or join a team.
  3. Comment on other projects.
  4. Come to the hacknight with your computer.
Abhiroop Sarkar
No Projects
1
Abhishek Gupta
No Projects
2
adarshkkumar
No Projects
3
Ajit Pai
No Projects
4
Amrit Raj Misra
No Projects
5
arun
No Projects
6
7
Ashwath
No Projects
8
Ashwin Acharya
No Projects
9
Asif
No Projects
10
Awase Khirni Syed
No Projects
11
Balaji Setty
No Projects
12
Bharat Shetty Barkur
No Projects
13
Chandra Reddy
No Projects
14
15
deepak chandramouli
No Projects
16
Deepu Thomas Philip
No Projects
17
Developer at Works
No Projects
18
Durga Prasad Mangali
No Projects
19
Eshan Mushtaq
1 Project
20
Faheem Mohammad
No Projects
21
Haris Ibrahim K. V.
No Projects
22
Harsh Kumar
No Projects
23
Harshit Jain
No Projects
24
himanshu garg
No Projects
25
Jagadish
No Projects
26
Jamna Vyas
No Projects
27
jogeswar meher
No Projects
28
29
Kumar Varun Varunesh
No Projects
30
Lokesh Jaiswal
No Projects
31
Madhur Garg
No Projects
32
Madhusudan
No Projects
33
Mahesh Garg
No Projects
34
Manuj Paliwal
No Projects
35
Mitesh Ashar
No Projects
36
Mohit Sharma
No Projects
37
Monit Singh
1 Project
38
Mrinal Kumar
No Projects
39
Mukesh G
No Projects
40
niket kumar
No Projects
42
nimish gupta
No Projects
43
Niraj Rajmohan
No Projects
44
Nitin Kumar
1 Project
45
Omkar Sharma MN
No Projects
46
Pavan Sudheendra
No Projects
47
Praseetha KR
No Projects
48
Prashanth Babu
No Projects
49
Prashanth Raghu
No Projects
50
Prateek Jain
No Projects
51
Pratik Mandrekar
No Projects
52
Pravin
No Projects
53
Ramana Reddy
No Projects
54
RamaVattigunta
No Projects
55
Ratnadeep Debnath
No Projects
56
Rohit Chatterjee
No Projects
57
Rommel Sharma
No Projects
58
SARDA VINAY KUMAR
No Projects
59
Shashi Shekhar Singh
No Projects
60
Shruti Ranade
No Projects
61
Sidharth N Kashyap
No Projects
62
Smita Sasindran
No Projects
63
Somanath
No Projects
64
Sujith Varghese
No Projects
65
Sunu Engineer
No Projects
66
67
68
Swaroop Krothapalli
No Projects
69
t3rmin4t0r
3 Projects
70
V. Sairanganath
No Projects
71
Vaishnavi Verma
No Projects
72
Varun Dhussa
1 Project
73
Varun Kulkarni
No Projects
74
Varun Torka
No Projects
75
Vinayak Hegde
No Projects
76
Vipul Mathur
No Projects
77
Vishnu V Ram
No Projects
78
Yogesh Sachwani
No Projects
79

Other Participants

abhijit mondal
No Projects
1
Arun S A G
No Projects
2
Bishan Singh Kochher
No Projects
3
Chintan Parekh
1 Project
4
Deepesh Shetty
No Projects
5
Deepesh Shetty
No Projects
6
Iliyas Shirol
No Projects
7
Krishna
No Projects
8
mahesh gondi
No Projects
9
Manasvi Gupta
No Projects
10
PANKAJ
No Projects
11
Ramesh
No Projects
12
Ranjeet Kumar
No Projects
13
sathya narayana Raju
No Projects
14
Sawani Bade
No Projects
15
shrikanth
No Projects
16
Sugandha Naolekar
No Projects
17
Sundar
No Projects
18
thamizhannal
No Projects
19
Uma Maheswararao
No Projects
20
Vikas Singh
No Projects
21
Yash - ConfusedCoder
No Projects
22