Microsoft Hadoop HDInsight ( Hadoop on Windows Azure)

-
Windows Azure HDInsight Service, formerly known as Hadoop on Windows Azure,  is now available inside the Windows Azure Preview portal.  Hadoop-based big data tools are what I call the WMD(P), or Weapons of Mass Data Processing. (You heard it here first!)  This is a very exciting development, and I would like to take a moment to recognize the great work our HDInsight team has done to pull this off.

Signing up

1. To sign up, log into your Windows Azure account at http://www.windowsazure.com.  At the bottom of the portal, click on New (+).
image
2. Then click on DATA SERVICES followed by HDInsight.  Use the preview program link to navigate to sign up page.

image

At the Preview features page, also accessible from https://account.windowsazure.com/PreviewFeatures/, click on try it now next to Azure HDInsight Preview. Please be warned that this might take up-to a few days.
image


Getting Started and more Learning Content

Overview presentation

Or a download from a direct link: http://bit.ly/YPstrp
Hands-on Lab
 New channel 9 Video Series

Direct links to the Windows Azure Portal Documentation Section

Quick Start

Visit these articles first to get started using the HDInsight Service.
Tutorial: Getting Started with the Windows Azure HDInsight Service
Tutorial: Using MapReduce with HDInsight
Tutorial: Using Hive with HDInsight
Tutorial: Using Pig with HDInsight

Explore

Guidance: Introduction to the Windows Azure HDInsight Service
A conceptual overview of the components of HDInsight.
Tutorial: Get Started with the HDInsight Service
Learn the fundamentals of working with the HDInsight service.

Plan

Guidance: What version of Hadoop is in Windows Azure HDInsight?
Learn what Hadoop components and versions are included in HDInsight.
Guidance: HDInsight Interactive JavaScript and Hive Consoles
HDInsight comes with interactive consoles for JavaScript and Hive that you can use as an alternative to remoting into the head node of a Hadoop cluster. Learn about how you can use the consoles to enter expressions, evaluate them, and then query and display the results of a MapReduce job immediately.

Upload

How to: Upload data to HDInsight
Learn how to upload and access data in HDInsight using Azure Storage Explorer, the interactive console, the Hadoop command line, or Sqoop.

Analyze

Tutorial: Connect Excel to Windows Azure HDInsight via HiveODBC
One key feature of Microsoft’s Big Data Solution is solid integration of Apache Hadoop with Microsoft Business Intelligence (BI) components. A good example of this is the ability for Excel to connect to the Hive data warehouse framework in the Hadoop cluster. This topic walks you through using Excel via the Hive ODBC driver.
Tutorial: Simple recommendation engine using Apache Mahout   
Apache Mahout is a machine learning library built for use in scalable machine learning applications. Recommender engines are some of the most immediately recognizable machine learning applications in use today. In this tutorial you use the Million Song Dataset site and download the dataset to create song recommendations for users based on their past listening habits.
Tutorial: Analyzing Twitter Data with Hive   
In this tutorial you will query, explore, and analyze data from Twitter using the Apache Hadoop-based HDInsight Service for Windows Azure and a complex Hive example.
Tutorial: Using HDInsight to process Blob Storage data and write the results to a SQL Database
This tutorial will show you how to use the HDInsight service to process data stored in Windows Azure Blob Storage and move the results to a Windows Azure SQL Database.

Manage

How to: Administer HDInsight
Learn how to create an HDInsight cluster and use the administrative tools available through the Windows Azure Management Portal.
How to: Monitor HDInsight
Learn how to monitor an HDInsight cluster and view Hadoop job history through the Windows Azure Management Portal.
How to: Deploy an HDInsight Cluster Programmatically
Learn how to use the Windows Azure service management REST API to create, list, and delete HDInsight clusters programmatically.
How to: Execute Remote Jobs on Your HDInsight Cluster Programmatically
Learn how to use the WebHCat REST API to provide metadata management and remote job submission to your Hadoop cluster.

Comments

Post a Comment

Popular posts from this blog

SharePoint 2007 - Simple Task Dashboard

-
Image
example of creating a simple Task Dashboard. This is a pretty basic example of using all OOTB (out of the box) features and combining them to quickly build a solution. Scenario: We want to provide a way for users to easily see a summary of the current project tasks. We want them to be able to see the following information: Open Tasks Assigned to Them Overdue Tasks Tasks Due This Week Task Calendar Tasks Assigned to Others Solution: To build this solution we will be using several different tools available in SharePoint, all included in the list below: Task List Task List Views We will create views for each of the different ways we want to see the data displayed on our dashboard. Web Part Page List View Web Parts Custom Navigation The first thing we will do is create our task lists and then configure all the views that we need. To create the list, go to View All Site Content, Create and then select the Task list. Next, we will need to configure our custom views. To do this, w
Read more

MERGE transformation in SSIS

-
Image
In this post we are gonna discuss about MERGE transformation. MERGE in SSIS is equal to UNION ALL in SQL Server. This transformation unions two datasets/tables.The merge Transformation combines two  sorted  dataset into single dataset. Highlighted the text SORTED in last statement because “It is not possible to use MERGE when the inputs are NOT SORTED”. There is one more transformation which is very similar to this i.e UNION ALL. The Merge Transformation is similar to the union all transformations. Use the union all transformation instead of the merge transformation in the following situations. The transformation inputs are not sorted The combined output does not need to be sorted. The transformation has more than two inputs. MERGE takes ONLY TWO inputs where as UNION ALL can take more than two. Now lets see how to configure MERGE transformation with an example. I created TWO tables with names MergeA and MergeB and inserted few records into each table as shown below.
Read more