Machine Learning Intro
Machine Learning is not only a tech-company kind of thing. Companies that are not traditionally tech companies are now incorporating machine learning too. No longer is machine learning just used by the Googles of the world. There are many organizations that are eager to implement machine learning and that’s where you come in.
When I was in college, the machine learning courses focused on MATLAB. Now, the focus is on Python. Learning Python is pretty critical nowadays in understanding and implementing Machine Learning. The specific packages that you should know is:
- NumPy
- SciPy
- Scikit-learn
- Theano
- TensorFlow
- Keras
- PyTorch
- Pandas
But before we get into this, what is machine learning? Arthur Samuel stated that Machine Learning is “the field of study that gives computers the ability to learn without being explicitly programmed.” That is an interesting statement in itself. Arthur Samuel built a checkers playing program that through self-play, the computer learned which moves led to more wins over losses. The program even became better than Arthur himself. In other words, the computer program became better than the programmer.
Today, machine learning algorithms outperform humans in many narrow tasks, such as playing checkers. And this isn’t even a surprise to us.
Tom Mitchel stated the following definition: “a computer program is said to learn from experience E with respect to some task T and some performance measure P, if its performance on T, as measured by P, improves with experience E.” A heck of a definition. What does it mean?
When applying this definition to checkers, experience E is referred to the computer program playing checkers against itself. The task T is pretty simple: it’s simply the task of playing checkers. The performance measure P might include defining what the chance of the computer program is in winning the game of checkers when it plays another opponent.
So, how do you utilize machine learning to implement such definitions. There are many different tools in order to do so. One such tool is called supervised learning. There are a few different categories of machine learning tools, but we’ll start with this one. The most widely used machine learning tool is supervised learning.
An example of machine learning is the relationship of a home’s price to the home’s square footage.
With supervised learning, if you’re given a dataset like this, how can you find the relationship between X (price) and Y (Sq. Ft). This might be the dataset for your city. If you’re trying to figure out exactly what to sell it for, you might draw a horizontal line from your sq. ft. axis and see which intersection point, price, it closely resembles. How do we get that intersection? By drawing a line that closely resembles the trend-line.
So, for example, for $180,000, you can get a house with approximately 3000 square feet. In this instance, we might want to fit a quadratic function instead since it might fit the data more accurately.
The example we just looked at is a type of a regression problem. The term regression refers to the fact that the y-value that we’re trying to predict is continuous.
What if we had a value of either 0 or 1? For example, what if we wanted to see if the home sold based on a particular price?
Can we figure out whether the home will sell based on a particular home’s price? This is an example of a classification problem. The classification refers to the y taking on a discrete number of variables. In regression, y is a real number, but in classification you have a finite number of answers. In the example above, y can be one of two possible outcomes: 0 or 1. There doesn’t need to be only 2 possible outcomes. The outcome can be any number of outcomes, as long as it’s finite.
With the classification problem above, we can map everything onto a horizontal line.
I’ve removed the 0’s from the dataset, but you can plot them as well, listing 0s as 0 and 1s as maybe x on the horizontal line graph. In both of the examples, the input x is a single real number. In most machine learning algorithms, you’re given multiple numbers to predict another number.
If you’re given two datasets, like the listing price and the length of months on the market, we might then see a chart where it shows whether the home was sold in the next 3 months.
Based on the dataset that you have, you’re trying to predict what the likelihood of selling a home is based on two inputs. For example, let’s say that the home has been on the market just over 7 months and the price is somewhere around $165,000. What is the likelihood that the home will sell in the next 3 months?
Looking at the dataset, you can probably quickly see that the likelihood is relatively high that the home will sell in the next 3 months.
We might even draw a line that shows the area of the graph where the likelihood of a sale is the highest.
To the left of the line, there’s a high probability that the sale will occur in the next 3 months, whereas data plotted to the right of the line will yield lower probability of a sale occurring in the next 3 months.
Normally, you have more than 2 features. For the sake of simplicity, we chose two, but usually there are so many features that you can’t reasonably plot them on a 2-dimensional graph. For example, we could have added:
- Square Footage
- Zip Code
- A+ School District
- Year Made
As you can see, the number of features can increase exponentially. You’ll learn about an algorithm called the Support Vector Machine that uses an infinite amount of features to produce an output. The Support Vector Machine uses an infinite dimensional vector to represent a home in this instance. You might be asking yourself whether this is even possible? We’ll touch upon SVM’s later, but for now know that SVM’s are supervised machine learning algorithms that are used for classification but can be used for regression. The more information you have about a subject, the better the learning algorithm will be in providing you the answer that you need.
With supervised learning, the idea is that during the learning stage, you’re given both the x-input and y-output, so that when you’re given a new x, you can predict a reasonable y.
Understanding Machine Learning algorithms is one thing, but knowing which ones to use, how to apply them, whether to collect more data or use a different algorithm, is a skillset that needs to be developed. Even though machine learning algorithms look like magic, there’s actually a systematized approach that can be applied to selecting the appropriate machine learning algorithm.
So far, everything that we’ve looked at revolved around supervised learning. However, if there’s supervised learning, then there must be something called unsupervised learning as well. What is unsupervised learning then? In the sales likelihood based on months on market and price of home example, we used supervised learning since we provided the input x and the output y.
In unsupervised learning, the algorithm is given just the inputs x and is then asked to find something interesting in the data. Some common unsupervised learning algorithms are:
- K-means Clustering
- Hierarchical Clustering
- Anomaly Detection
- Principal Component Analysis
- Apriori Algorithm
Most economic impact is made through supervised learning, but there are use-cases for unsupervised learning as well.
Anything else? How about reinforcement learning? When you don’t know the optimal way of doing something, like driving, then you can utilize reinforcement learning to first let the vehicle try out a few different ways to drive and then when it drives properly, you reward it, and when it doesn’t you either don’t reward it or you specify that the action was not desired. With that type of reinforcement, the reinforcement algorithm then tries to figure out how to drive a car properly.
Dino Cajic is the CEO at MyAutoSystem. He’s also currently the Head of IT at Absolute Biotech, the parent company of LSBio (LifeSpan BioSciences, Inc.), Absolute Antibody, Kerafast, Everest BioTech, Nordic MUbio and Exalpha. He has a B.S. in Computer Science, a minor in Biology, and over a decade of software engineering experience. His background consists of creating enterprise level e-commerce applications, performing research based software development, and facilitating the spread of knowledge through writing.
You can connect with him on his website or his LinkedIn, follow him on Instagram, or subscribe to his Medium publication.
