Machine Learning Interview Questions (Part 2)

This is 2nd part “Machine Learning Interview Questions”. To read the first part of this series, click here

What are the pros and cons (advantages and disadvantages) of Bayes’ Theorem?

Pros: 1. Bayes’ theorem is relatively simple to understand and build, 2. We can train it easily; even with a small dataset, 3. It’s fast!, 4. It’s not sensitive to irrelevant features. Cons: 1. It assumes every feature is independent, which isn’t always the case

Read More

What is the difference between L1 and L2 regularization?

The difference between the L1 and L2 is just that L2 is the sum of the square of the weights, while L1 is just the sum of the weights.

Read More

Briefly describe Cluster analysis

Cluster analysis or clustering is the task of grouping a set of objects in such a way that objects in the same group (called a cluster) are more similar (in some sense or another) to each other than to those in other groups (clusters). It is a main task of exploratory data mining, and a common technique for statistical data analysis, used in many fields, including machine learning, pattern recognition, image analysis, information retrieval, bioinformatics, data compression, and computer graphics.

Read More

What is ICA or Intependent Component Analysis?

In signal processing, independent component analysis (ICA) is a computational method for separating a multivariate signal into additive subcomponents. This is done by assuming that the subcomponents are non-Gaussian signals and that they are statistically independent from each other. ICA is a special case of blind source separation. A common example application is the “cocktail party problem” of listening in on one person’s speech in a noisy room.

Read More

What is deep learning?

Deep learning, a subset of machine learning, utilizes a hierarchical level of artificial neural networks to carry out the process of machine learning. The artificial neural networks are built like the human brain, with neuron nodes connected together like a web. While traditional programs build analysis with data in a linear way, the hierarchical function of deep learning systems enables machines to process data with a non-linear approach. A traditional approach to detecting fraud or money laundering might rely on the amount of transaction that ensues, while a deep learning non-linear technique to weeding out a fraudulent transaction would include time, geographic location, IP address, type of retailer, and any other feature that is likely to make up a fraudulent activity.

Read More

Where can we use deep learning?

1. Automatic speech recognition, 2. Image recognition, 3. Visual Art Processing, 4. Natural language processing, 5. Drug discovery and toxicology, 6. Customer relationship management, 7. Recommendation systems, 8. Bioinformatics, 9. Mobile Advertising

What is random forest?

Random forests or random decision forests[1][2] are an ensemble learning method for classification, regression and other tasks, that operate by constructing a multitude of decision trees at training time and outputting the class that is the mode of the classes (classification) or mean prediction (regression) of the individual trees. Random decision forests correct for decision trees’ habit of overfitting to their training set.

Read More

What do you know about Dimensionality Reduction Algorithms?

In machine learning and statistics, dimensionality reduction or dimension reduction is the process of reducing the number of random variables under consideration,[1] via obtaining a set of principal variables. It can be divided into feature selection and feature extraction.

Read More

What are the popular programming languages used in machine learning?

Python, Java, R, Scala etc

What are the popular frameworks in machine learning?

1. Apache Spark MLlib, 2. TensorFlow, 3. Amazon Machine Learning (AML), 4. Apache Singa, 5. Torch, 6. Azure ML Studio etc

What is tensor flow?

TensorFlow is an open-source software library for dataflow programming across a range of tasks. It is a symbolic math library, and also used for machine learning applications such as neural networks.[3] It is used for both research and production at Google.

What is data mining?

Data mining is the process of sorting through large data sets to identify patterns and establish relationships to solve problems through data analysis. Data mining tools allow enterprises to predict future trends.

Read More

How do you avoid overfitting with a model?

Use test data for evaluation or do cross validation. Add regularizations terms (such as L1, L2, AIC, BIC, MDL or a probabilistic prior) to the objective function.

Read More

What is Kernel Trick?

In machine learning, kernel methods are a class of algorithms for pattern analysis, whose best known member is the support vector machine (SVM). The general task of pattern analysis is to find and study general types of relations (for example clusters, rankings, principal components, correlations, classifications) in datasets. kernel methods require only a user-specified kernel, i.e., a similarity function over pairs of data points in raw representation.

Read More

When will machines eat us?

Answer based on your own preference

Will machines ever be able to feel consciousness. What do you think?

Answer based on your own preference

What is apache spark?

Apache Spark is a powerful open source processing engine built around speed, ease of use, and sophisticated analytics. It was originally developed at UC Berkeley in 2009

Read More

What is the difference between hash table and array?

1) Hash table store data as name, value pair. While in array only value is store. 2) To access value from hash table, you need to pass name. While in array, to access value, you need to pass index number. 3) you can store different type of data in hash table, say int, string etc. while in array you can store only similar type of data.

Read More

What are some of the major tasks in data pre-processing?
  1. Data cleaning: Fill in or missing values, detect and remove noisy data and outliers.
  2. Data transformation: Normalize data to reduce dimensions and noise.
  3. Data reduction: Sample data records or attributes for easier data handling.
  4. Data discretization: Convert continuous attributes to categorical attributes for ease of use with certain machine learning methods.
  5. Text cleaning: remove embedded characters which may cause data misalignment, for e.g., embedded tabs in a tab-separated data file, embedded new lines which may break records, etc.

Read More

How to deal with missing values?

To deal with missing values, it is best to first identify the reason for the missing values to better handle the problem. Typical missing value handling methods are: Deletion: Remove records with missing values Dummy substitution: Replace missing values with a dummy value: e.g, unknown for categorical or 0 for numerical values. Mean substitution: If the missing data is numerical, replace the missing values with the mean. Frequent substitution: If the missing data is categorical, replace the missing values with the most frequent item Regression substitution: Use a regression method to replace missing values with regressed values.

Read More

Machine Learning Interview Questions (Part 1)

Nearly all big tech companies have an artificial intelligence project, and they are willing to pay experts millions of dollars to help get it done. – By CADE METZ

Machine learning is a part of artificial intelligence. According to IBM’s forecast, job opening for artificial intelligence, machine learning and data science will increase 28% by 2020 (Forbes).

So if you are looking for a machine learning job or need to prepare for machine learning interview, then take a look at following questionaries.

What is machine learning?

Machine learning is a branch of Artificial Intelligence. It allows systems to automatically learn and improve from experience without being explicitly programmed.

What is artificial intelligence?

Artificial Intelligence is a branch of Computer Science that studies and researches to develop machines that have intelligence like human being. Most importantly, they can learn from experience and deal with new situations smartly.

What is the difference between artificial intelligence and machine learning?

Artificial Intelligence (AI) has many branches. One of them is ML. AI deals with broader context of developing a machine that can act like human and smartly. On the other hand, in machine learning we provide data to machines and they learn for themselves from that data.

What are the types of machine learning?

There are 3 types of machine learning. 1. Supervised learning, 2. Unsupervised learning and 3. Reinforced learning

What is Supervised machine learning?

In supervised machine learning, you provide a set of data with problems and answers. Machine learns from that set of data and applies learning in future.

What is Unsupervised machine learning?

In unsupervised learning, we don’t provide any solution data to machine. We provide them a set of data. The machine learns for itself.

What is Reinforcement machine learning?

Reinforcement learning is training by rewards and punishments. Here we train a computer as if we train a dog. If the dog obeys and acts according to our instructions we encourage it by giving biscuits or we punish it (by not providing biscuit or any other mean). Similarly, if the system works well then the teacher gives positive value (i.e. reward) or the teacher gives negative value (i.e. punishment). The learning system which gets the punishment has to improve itself. Thus it is a trial and error process.

Read More

What are the algorithms used in machine learning?

1. Linear Regression, 2. Logistic Regression, 3. Decision Tree, 4. SVM, 5. Naive Bayes, 6. KNN, 7. K-Means, 8. Random Forest, 9. Dimensionality Reduction Algorithms, 10. Gradient Boosting algorithms, 10.1. GBM, 10.2. XGBoost, 10.3. LightGBM, 10.4. CatBoost

Explain Linear Regression

Linear regression is a statistical method that attempts to model relationship between different scalar variables. There can be two or more variables. Among them, one is dependent variable. Others are independent variables.

What do you know about logistic regression?

Like all regression analyses, the logistic regression is a predictive analysis. Logistic regression is used to describe data and to explain the relationship between one dependent binary variable and one or more nominal, ordinal, interval or ratio-level independent variables.

What is the difference between linear regression and correlation?

From correlation we can only get an index describing the linear relationship between two variables; in regression can predict the relationship between more than two variables and can use it to identify which variables x can predict the outcome variable y. … While regression means going back towards average .

Read More

When to use decision tree vs logistic regression?

A logistic regression model is searching for a single linear decision boundary in your feature space, whereas a decision tree is essentially partitioning your feature space into half-spaces using axis-aligned linear decision boundaries. The net effect is that you have a non-linear decision boundary, possibly more than one.

This is nice when your data points aren’t easily separated by a single hyperplane, but on the other hand, decisions trees are so flexible that thIt depends on your specific problem and the data you have. Both decision trees (depending on the implementation, e.g. C4.5) and logistic regression should be able to handle continuous and categorical data just fine. ey can be prone to overfitting. To combat this, you can try pruning. Logistic regression tends to be less susceptible (but not immune!) to overfitting.

Lastly, another thing to consider is that decision trees can automatically take into account interactions between variables, e.g. xyxy if you have two independent features xx and yy. With logistic regression, you’ll have to manually add those interaction terms yourself.

Which algorithms do we use for supervised machine learning?

Classification Algorithms: 1. Support vector machines (SVM), 2. Neural networks, 3. Naïve Bayes classifier, 4. Decision trees, 5. Discriminant analysis, 6. Nearest neighbors (kNN); Regression Algorithms: 1. Linear regression, 2. Nonlinear regression, 3. Generalized linear models, 4. Decision trees, 5. Neural networks

Read More

Which algorithms do we use for unsupervised machine learning?

a. Clustering: k-means, mixture models, hierarchical clustering, b. Neural Networks: Hebbian Learning, Generative Adversarial Networks. c. Approaches for learning latent variable models: Expectation–maximization algorithm (EM) Method of moments.

Read More

How is KNN different from k-means clustering?

K-nearest neighbors is a classification algorithm, which is a subset of supervised learning. K-means is a clustering algorithm, which is a subset of unsupervised learning. … In sum, two different algorithms with two very different end results

Read More

What is ROC curve and how it works?

In statistics, a receiver operating characteristic curve, i.e. ROC curve, is a graphical plot that illustrates the diagnostic ability of a binary classifier system as its discrimination threshold is varied. … ROC analysis is related in a direct and natural way to cost/benefit analysis of diagnostic decision making.

Read More

What is semi supervised machine learning?

Semi-supervised machine learning is a mixure of supervised learning and unsupervised learning. Here, some data is labeled but most of it is unlabeled.

Read More

What is Ordinary Least Squares Regression?

In statistics, ordinary least squares (OLS) or linear least squares is a method for estimating the unknown parameters in a linear regression model, with the goal of minimizing the sum of the squares of the differences between the observed responses (values of the variable being predicted) in the given dataset and those predicted by a linear function of a set of explanatory variables.

Read More

Briefly describe Naïve Bayes Classification

Naive Bayes is a collection of classification algorithms based on Bayes Theorem. It is not a single algorithm but a family of algorithms that all share a common principle, that every feature being classified is independent of the value of any other feature. So for example, a fruit may be considered to be an apple if it is red, round, and about 3″ in diameter. A Naive Bayes classifier considers each of these “features” (red, round, 3” in diameter) to contribute independently to the probability that the fruit is an apple, regardless of any correlations between features. Features, however, aren’t always independent which is often seen as a shortcoming of the Naive Bayes algorithm and this is why it’s labeled “naive”.

Read More

Do you know the meaning of SVM?

“Support Vector Machine” (SVM) is a supervised machine learning algorithm which can be used for both classification or regression challenges. However, it is mostly used in classification problems. In this algorithm, we plot each data item as a point in n-dimensional space (where n is number of features you have) with the value of each feature being the value of a particular coordinate. Then, we perform classification by finding the hyper-plane that differentiate the two classes very well.

Read More

 

What is Machine Learning

Machine Learning
IBM Cognitive Hypervisor uses machine learning and cognitive computing to analyze medical data for real-time care

Summary: If you are not living in a cave or rural area, you must have heard the word “Machine Learning”. Although it is a term more related to “Computer Science”, people from other fields are also interested about it. So what is machine learning?

It is a branch of Artificial Intelligence (AI). It deals with huge amount of data. Different statistical algorithms and formula are applied to data to find pattern in the dataset or get a result. Simply speaking, it is a set of algorithms plus statistics.

Details: Machine learning is a buzz word of modern age. Literally it means, machines can learn from experience or data. In fact, machine learns more, if you provide more data.

Why machine learning is so popular?

Because if you can correctly choose right algorithm and statistical formula, then you can predict many things. For example, weather condition of future (Okay, near future), customer’s buying trend, share market movements etc.

What you need to learn to start machine learning?

  1. Hardware: from single computer to cluster of inter-connected computers. Or, amazon’s EC2 instances.
  2. Some machine learning algorithms
  3. Some Statistics formulas
  4. Framework (Tensorflow or deeplearning4j etc)
  5. Any one programming language – python, java, scala.

Photo Credit:

IBM Research

 

Types of Machine Learning : Supervised and Unsupervised

Machine Learning Types
“Machine Learning” can categorize objects into groups and find patterns

When it’s time to learn “Machine Learning”, the first thing that you will hear is “Types of Machine Learning”. Because this is where you will begin to learn.

Based on learning algorithms, machines can learn in two ways. In supervised way and the other is un-supervised way. So these are the types of machine learning. Let’s discuss about them with examples.

1. Supervised machine learning

Here at the very beginning you teach your machine. Then the machine gives you result based on your  lessons. Let me give you a real world example:

Suppose, you want to teach the machine to recognize images of fruits. In supervised learning process you show the image of apple and tell machine that this is apple. Again you take image of orange and let it know that the image contains orange.

By this way you teach your machine with lot of images and their labels.

After that, if you show a new image to the machine, most likely it will recognize the fruit’s name.

2. Unsupervised machine learning

Here you don’t teach machine. It learns itself. Lets jump into an example. In this scenario,  you show many images of apples and oranges. But don’t say which one is apple and which one is orange. The machine will be able to predict that these two things are different. It will categorize apple in one category and orange in another category.

That is, it will cluster different things in different groups.

Isn’t it that simple?

Photo Credit:

Army Photography Contest – 2007

Big Data for Banking Sector

In the field of nanotechnology, scientists of different fields such as physics, chemistry and biology work under the same roof. By the same way, in the field of Big Data, not only programmers, also statisticians, decision makers, system admins all work together. Big data is taking our modern computer based society to a new height. Still it’s the beginning of the journey and has started to reveal it’s possibility to different fields, such as social network, marketing, health care, law enforcement as well as banking and insurance Sector. Today we’ll explore the possibilities of big data in banking and financial sector.

Why Big Data in Banking Sector?

Big data deals with…
1. large amount of data (Volume),
2. coming from different sources and in different formats such as transaction data, customer photo, CCTV images (Variety)
3. that needs to be processed very fast (velocity).

Banking data has all these 3 properties. That’s why big data fits in banking sector.

There are 5 major areas of implementing Big Data in Banking sector:
1. Fraud Detection
2. Analyze customer sentiment and feedback
3. Analyze transaction history to discover trends around the year
4. Analyze customer’s spending history and behavior to offer personalized product
5. Customer Recognition

1. Fraud detection:
Currently banks detect frauds depending on ip address, unusual login time and “using wrong pin multiple time” etc. Big data changed the process of fraud detection. By analyzing large amount of data, applying algorithm and fraud pattern, it has become possible to detect and prevent frauds or unusual activity instantly.

2. Analyze customer sentiment and feedback:
It is easy to get new customers. But retaining them for long time is challenging. So it is important to analyze customer’s complain, suggestion, satisfaction and make decision based on analysis result. For example improving a service that is performing poor and dissatisfying customer, relocating ATM booth etc.

3. Analyze transaction history to discover trends
Banks have large amount of transaction history. By analyzing those data, we can discover trends of spending and savings of customers. We can find when and where users spend more amount of money. For example, before festivals customers spend more in shopping malls. By discovering these trends banks can offer better services for it’s all customers.

4. Analyze customer’s spending history and behavior
In this case, banks analyze single customer’s transaction history and behavior. This analysis helps management of banks to offer personalized products. For example, banks can offer dual currency products with benefits to the customers who travel frequently abroad. Banks can also offer special discounts to customers who make online transactions more often.

5. Customer Recognition: It is time consuming to recognize a customer when he or she goes to bank to withdraw money or perform other transaction operation. By using realtime analysis power of Big data, banks can analyze photo, signature, voice and behavior to recognize customer. This can save time and let bank stuffs server more customers.

Useful links:

Deriving Business Insight from Big Data in Banking

Big Data in Financial Services and Banking