Skip to main content

Azure Hadoop HDInsight Tools – Wrote A VS Add In For LINQ To Hive Model Generation and Hive SQL Queries

imageThis post is about the HD Insight Tools Visual Studio Add In -  for accessing Azure Hadoop Hive  from Visual Studio. I hacked together the same on my way back from the MVP summit, and a post is due. If you are new, last week I wrote about issuing LINQ queries against the Hive meta store – read that post for a general introduction about LINQ to Hive.

You may use HDInsight Tools Visual Studio Add In for

  • Generating wrapper classes for Hive tables for easily executing LINQ to Hive queries.
  • Running Hive SQL queries from Visual Studio and for viewing the results

Please note that you need to provision a Hadoop cluster in Azure, or need HDInsight installed locally (See these earlier posts related to Hadoop on Azure - Analyzing some ‘Big Data’ using C# and Extracting Top 500 MSDN Links from Stack Overflow ).

Installing The Add In

You can install the Add In from Visual Studio Tools –> Extensions and Updates.

image

 

Generating LINQ To Hive Model Classes

Once you install the Add In, Goto View-> Other Windows –> Hive Explorer to bring up the Hive Explorer tool Window. Click Add Connection to add a hive connection, by providing the server details, port, user name and password and click Save.

image

Once you add a connection, you can expand the connection to view the tables and fields in the Hive. Now, you can right click on the Added connection name in the tool Window, and click ‘Generate Code’ context menu to generate the model classes. Also, make sure you install the Microsoft.Hadoop.Hive nuget package by issuing the install-package Microsoft.Hadoop.Hive –pre nuget command.

image

You can use the generated model to query Hive using LINQ as mentioned in the previous article.

Issuing Queries

You can use the Queries window of Hive Explorer to issue Hive Queries. Choose the connection in the combo box, and shoot your query to view the results.

image

Happy Coding. I’m soon uploading the source code of Hive Explore to Github after some clean up once I’ve some time. Smile

Popular posts from this blog

Top 7 Coding Standards & Guideline Documents For C#/.NET Developers

Some time back, I collated a list of 7 Must Read, Free EBooks for .NET Developers, and a lot of people found it useful. So, I thought about putting together a list of Coding Standard guidelines/checklists for .NET /C# developers as well.As you may already know, it is easy to come up with a document - the key is in implementing these standards in your organization, through methods like internal trainings, Peer Reviews, Check in policies, Automated code review tools etc. You can have a look at FxCop and/or StyleCop for automating the review process to some extent, and can customize the rules based on your requirements.Anyway, here is a list of some good Coding Standard Documents. They are useful not just from a review perspective - going through these documents can definitely help you and me to iron out few hidden glitches we might have in the programming portion of our brain. So, here we go, the listing is not in any specific order.1 – IDesign C# Coding StandardsIDesign C# coding stand…

Creating a quick Todo listing app on Windows using IIS7, Node.js and Mongodb

As I mentioned in my last post, more and more organizations are leaning towards Web Oriented Architecture (WOA) which are highly scalable. If you were exploring cool, scalable options to build highly performing web applications, you know what Node.js is for.After following the recent post from Scott Hanselman, I was up and running quickly with Node.js. In this post, I’ll explain step by step how I’ve setup Node.js and Mongodb to create a simple Todo listing application.Setting up Node.jsThis is what I’ve done.1 – Goto http://nodejs.org/, scroll down and download node.exe for Windows, and place it in your c:\node folder2 – Goto IIS Node project in Git at https://github.com/tjanczuk/iisnode, download the correct ‘retail’ link of IIS Node zip file (I downloaded the already built retail package, otherwise you can download and build from the source).3 – Extract the zip file some where, and run the install.bat or install_iisexpress.bat depending on your IIS Version. If you don’t have IIS in…

5 Awesome Learning Resources For Programmers (To help you and your kids to grow the geek neurons)

Happy New Year, this is my first post in 2012. I’ll be sharing few awesome learning resources I’ve bookmarked, and will be pointing out some specific computer/programming related courses I've found interesting from these resources.Also, thought about saving this blog post for my kids as well - instead of investing in these Child education schemes (though they are too small as of today, 2 years and 60 days respectively ). Anyway, personally my new year resolution is to see as much videos from this course collections (assuming I can find some free time in between my regular job && changing my babies diapers).1 – Khan AcademyAs I mentioned some time back, you and your kids are missing some thing huge if you havn’t heard about Khan Academy.  It is an awesome learning resource, especially if you want to re-visit your basics in Math, Science etc.With a library of over 2,600 videos covering everything from arithmetic to physics, finance, and history and 268 practice exercises, th…