Advice to an aspiring Data Scientist

If you have something in your mind “I want to be a data scientist when I grow up.”

My advice to you.

Be curious,
Extremely argumentative,
And Judgmental.

Curiosity is absolutely must.

I am professional astrologer also. I know if you have good Mercury, it means you have curiosity and want to learn.

Argumentative and Judgmental help you in start. It doesn’t matter right or wrong.

Ability to take position is important after curiosity.

Data would helps in learning and in process you modify your assumptions and hypothesis and helps to lead in conclusion.

Next thing as a data scientist, you would need is some comfort and flexibility with analytics platforms.

Next is the ability to tell a story.

Because after your analysis you have to tell the story that you got from data.

Vikas Jindal

 

Advertisements

Data Science

What is Data Science?

Data Science is a process where you extract live information from dead data.

It’s a branch of science where you learn from history in interesting way.

You can work for betterment in multi dimensions.

It’s a way of story telling with the help of data.

Data science came in picture today because we have tons of data available (Storage is cheap), we have cloud and we have processing power.

Vikas Jindal

 

Angular Module – My Notes

Angular modules are a key part of any Angular application. An Angular module is a class with an NgModule decorator.

An Angular module’s purpose:

  • To organize the pieces of our application,
  • Arrange them into cohesive blocks of functionality,
  • Extend our application with capabilities from external libraries.

Angular modules library the environment for resolving the directives and pipes in our components templates.

Modules are a great way to selectively aggregate classes from other modules, and re-export them in a consolidated convenience module. BrowserModule, HttpModule and RouterModule are all examples.

An Angular module can be loaded eagerly when the application starts or it can be lazy loaded asynchronously by the router.

An Angular module declares each component, directive and pipe that it manages. Every component, directive and pipe we create belongs to an Angular module.
An Angular module bootstraps our root application component.

Defining the component needed to display our first template. An Angular module can export components, directives, pipes and even other Angular modules, making them available for other modules to import and use.

An Angular module imports other Angular modules. This brings in the export functionality from those imported modules.

An Angular module can register service providers with the Angular injector, making the services available to any class in the application.

We can think of Angular module as a box. Inside that box we declare each of our components. If those components need any functionality that functionality also needs to be defined within this box.

Introduction of Angular

Angular 2 is

  • An open source
  • JavaScript client side framework for creating powerful web applications
  • Created and maintained by Google

Language

The Angular team uses TypeScript to build Angular.

Version

We know Angular Version 1 as AngularJs.

We know Angular Version 2 & 4 as  Angular.

Prerequisites

  • HTML
  • Javascript
  • Typescript
  • ES 2015
  • CSS
  • Ajax

Architecture Elements

  • Module
  • Component
    • Template
    • Class or Code
    • Metadata
    • Data Binding
  • Directive
  • Filter
  • Service

Angular Module

Modules help to organize Angular Application. A Angular Application can have many modules. But Angular Application has atleast a single Module – We call it, Root Module or App Module.

Module can be importing as library in other module.

Module has dedicated single purpose in Angular Application.

With Root Module, Angular Application can has other features and shared modules. Module can have one or more than one componet.

Angular Module = Component + Component + Component

Example of Angular Module

If we think about a project about school then we can divide project in following modules

  • Student
  • Teacher
  • Class
  • Subject

In Angular, We create above modules and shared modules.

I will discuss modules in coming post.

Component

Component is a building block in Angular Application. It has an interface and data. It is like User Control or ActiveX control. It contains a class, a decorative function and a template.

Component = Template + Class + Metadata

Example of Component

import { Component } from '@angular/core'
@Component ( {
selector: 'proj-app',
template: '
{{pageTitle}}
'
}) export class AppComponent { pageTitle: string = 'Project: Home Page by Vikas Jindal'; }

Template

Template is an interface or view in Angular. It contains HTML.

Example of Template

template: '

{{pageTitle}}

My First Angular Component

We can write HTML inline but when HTML code is large then we use seperate file and then we use “templateUrl” to link external file with component.

What is {{}}?

Here, a concept of an interpolation comes in picture and we can bind values. ‘{{}}’ are interpolation braces and uses in subsitution of properties, evalution of expressions and calling functions.

Class or Code

Class has data elements, properties and methods written in TypeScript. It supports View or template.

Metadata

Metadata is act as glue between HTML Template and Class or code. Without Metadata class is simple class. It defines with the help of decorator function.

By Vikas Jindal

Next: Angular 2 – Hello World

Angular 2 – Hello World

Firends, After introduction of Angular 2. Now we are going to write our first program in Angular 2.

=> Angular Application = AngApp

Refresh:

AngApp contains modules.

Module contains components.

Component conatins 3 parts.

  1. Template
  2. Class or Code
  3. Metadata

We need to install first NPM. You can download it from following URL ‘https://nodejs.org/en/download/’.

Start

Create folder for project.

mkdir firstproject
cd firstproject

Now we need to create following files in our project folder

file name: tsconfig.json

This is configuration file for TypeScript compiler.

{
  "compilerOptions": {
    "target": "es5",
    "module": "system",
    "moduleResolution": "node",
    "sourceMap": true,
    "emitDecoratorMetadata": true,
    "experimentalDecorators": true,
    "removeComments": false,
    "noImplicitAny": false
  },
  "exclude": [
    "node_modules",
    "typings/main",
    "typings/main.d.ts"
  ]
}

file name: typings.json

This is used to identify TypeScript definition files in AngApp.

{
  "globalDependencies": {
    "core-js": "registry:dt/core-js#0.0.0+20160602141332",
    "jasmine": "registry:dt/jasmine#2.2.0+20160621224255",
    "node": "registry:dt/node#6.0.0+20160621231320"
  }
}

 

file name: package.json

This file contains packages require in AngApp.

{
  "name": "angular2-demo",
  "version": "1.0.0",
  "scripts": {
    "start": "concurrent \"npm run tsc:w\" \"npm run lite\" ",
    "tsc": "tsc",
    "tsc:w": "tsc -w",
    "lite": "lite-server",
    "typings": "typings",
    "postinstall": "typings install"
  },
  "license": "ISC",
  "dependencies": {
    "angular2": "2.0.0-beta.7",
    "systemjs": "0.19.22",
    "es6-promise": "^3.0.2",
    "es6-shim": "^0.33.3",
    "reflect-metadata": "0.1.2",
    "rxjs": "5.0.0-beta.2",
    "zone.js": "0.5.15"
  },
  "devDependencies": {
    "concurrently": "^2.0.0",
    "lite-server": "^2.1.0",
    "typescript": "^1.7.5",
    "typings":"^0.6.8"
  }
}

Open npm command prompt and use following command to install npm packages.

npm install

If you get some errors then ingnore them.

Prepare project files

mkdir App
cd App

file name: firstproject.component.ts

This file contains code for component.

import {Component} from "angular2/core";

@Component({
   selector: 'my-app',
   template: '<h2>My First Angular 2 App</h2>
{{msg}}
' }) export class AppComponent { msg:string = "Hello World!"; }

file name: firstproject.main.ts

import {bootstrap} from "angular2/platform/browser"
import {AppComponent} from "./firstproject.component"

bootstrap(AppComponent);

Move to project folder

cd ..

and create final file

file name: index.html

<!DOCTYPE html>
<html>
  <head>
    <title>Hello World</title>
     src="https://cdnjs.cloudflare.com/ajax/libs/es6-shim/0.33.3/es6-shim.min.js">
     src="https://cdnjs.cloudflare.com/ajax/libs/systemjs/0.19.20/system-polyfills.js">
     src="https://code.angularjs.org/2.0.0-beta.6/angular2-polyfills.js">
     src="https://code.angularjs.org/tools/system.js">
     src="https://code.angularjs.org/tools/typescript.js">
     src="https://code.angularjs.org/2.0.0-beta.6/Rx.js">
     src="https://code.angularjs.org/2.0.0-beta.6/angular2.dev.js">
    
      System.config({
        transpiler: 'typescript',
        typescriptOptions: { emitDecoratorMetadata: true },
        packages: {'app': {defaultExtension: 'ts'}},
        map: { 'app': './app' }
      });
      System.import('app/firstproject.main')
            .then(null, console.error.bind(console));
    
  </head>
<body>
   <my-app>Loading...</my-app>
</body>
</html>

To complie and Run your firstproject. Type following command in command prompt

npm start

 

By Vikas Jindal

Atlas Serice is down

I was not able to execute a single command in Hive CLI and getting following error.

FAILED: Hive Internal Error: com.sun.jersey.api.client.ClientHandlerException(java.io.IOException: java.net.ConnectException: Connection refused)
com.sun.jersey.api.client.ClientHandlerException: java.io.IOException: java.net.ConnectException: Connection refused
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:149)
at com.sun.jersey.api.client.Client.handle(Client.java:648)
at com.sun.jersey.api.client.WebResource.handle(WebResource.java:670)
at com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74)
at com.sun.jersey.api.client.WebResource$Builder.method(WebResource.java:623)
at org.apache.atlas.AtlasClient.callAPIWithResource(AtlasClient.java:351)
at org.apache.atlas.AtlasClient.callAPIWithResource(AtlasClient.java:346)
at org.apache.atlas.AtlasClient.getType(AtlasClient.java:191)
at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.registerHiveDataModel(HiveMetaStoreBridge.java:496)
at org.apache.atlas.hive.hook.HiveHook.fireAndForget(HiveHook.java:194)
at org.apache.atlas.hive.hook.HiveHook.run(HiveHook.java:172)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1585)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1254)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1118)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1108)
at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:216)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:168)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:379)
at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:739)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:684)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:624)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by: java.io.IOException: java.net.ConnectException: Connection refused
at org.apache.atlas.security.SecureClientUtils$1$1.run(SecureClientUtils.java:107)
at org.apache.atlas.security.SecureClientUtils$1$1.run(SecureClientUtils.java:99)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.atlas.security.SecureClientUtils$1.getHttpURLConnection(SecureClientUtils.java:99)
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler._invoke(URLConnectionClientHandler.java:159)
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:147)
… 26 more
Caused by: java.net.ConnectException: Connection refused

Solution:

Start or restart Atlas Service from Ambari…

Install Python in Windows env

Install Virtual Environment

C:\> pip intall virtualenv

Create Virtual Environment

virtualenv test

Activate Virtual Environment

test\Scripts\activate

Result ….. (test) C:\>

Install Django in Virtual Environment

(test) C:\> pip install django