Time Series Forecasting: Predicting the Future with Patterns from the Past

Data Science · beginner

Featured image for Time Series Forecasting: Predicting the Future with Patterns from the Past

📈 Introduction: The Art of Prediction

Imagine this: You are the captain of a giant cargo ship. You need to sail across the ocean. To stay safe, you have to know what the weather will be like tomorrow. Will there be a storm? Will the sea be calm?

You don’t have a magic crystal ball. But you do have a massive chart showing the wind speed, temperature, and ocean currents over the last 20 years. You look at the patterns. You notice that every August, the wind speeds increase slightly. You use that past data to make a guess about what will happen tomorrow.

That process—looking at data collected over time to predict future time periods—is called Time Series Forecasting.

A Time Series is simply a sequence of data points collected at regular intervals: every hour, every day, every week, or every year.

  • The temperature in Singapore at 1:00 PM every day for 10 years is a Time Series.
  • The price of a PlayStation 5 every month since it was released is a Time Series.
  • The number of students in your school every year for the last 50 years is a Time Series.

In this 3000+ word guide, we are going to learn how AI and Machine Learning look at these historical patterns and make incredibly accurate predictions about tomorrow!

⏳ Chapter 1: The 3 Parts of Every Time Series

If you draw a Time Series on a graph (with Time on the bottom, and the measured number on the side), it looks like a wavy line going across the page. But hidden inside that wavy line are three distinct “signals” or components.

1. The Trend (The General Direction)

The Trend is the long-term direction of the graph. Is the data generally moving up, moving down, or staying flat?

  • Example: The temperature in Singapore is staying relatively flat (Trend). But the global average temperature is moving up over the last 50 years (Trend = Upward).

2. The Seasonality (The Repeating Cycle)

Seasonality is a pattern that repeats at fixed, regular intervals—usually because of seasons, holidays, or weekends.

  • Example: Every December, Singapore malls sell far more toys than in February. If you look at a graph of “Toy Sales,” you will see a massive spike every single December. That is Seasonality.
  • School Example: If you graph “School Canteen Sales,” you will see a repeating pattern every 5 days. Monday is usually busy with breakfast, Friday is busy with lunch. That’s weekly Seasonality.

3. The Residual / Noise (The Random Jumps)

Noise is the random, unpredictable wobble in the data. It is completely irregular and caused by weird, one-off events.

  • Example: A sudden power outage at the mall caused sales to drop to zero for 1 hour on a random Tuesday. That drop has nothing to do with the Trend or Seasonality—it’s just random Noise.
  • AI models are excellent at ignoring Noise. They try to filter it out so they can focus purely on the Trend and Seasonality to make the best predictions.

🤖 Chapter 2: How AI Predicts the Future (The Difference Between Regression and Forecasting)

You might be wondering: “Can’t a normal Machine Learning model just look at the data and predict?”

Not exactly. There is a big difference between predicting what something is and predicting what will happen next.

Regression (Predicting a Number)

In our Machine Learning article, we learned about predicting house prices. You feed the AI the “Features” (size, bedrooms, location) and it predicts the “Label” (price). The AI assumes that all the data points are independent. It doesn’t care if House A was sold in 1990 and House B was sold in 2020.

Time Series Forecasting (Predicting a Sequence)

Time Series is much harder because the data points are dependent on each other. You can’t predict tomorrow’s temperature without knowing what today’s temperature was. The sequence matters immensely.

How Time Series AI works

Instead of feeding the AI a list of features, we feed the AI the “Lag” values.

  • Lag 1 = Yesterday’s temperature.
  • Lag 2 = The day before yesterday’s temperature.
  • Lag 3 = The day before that’s temperature.

The AI looks at the Lags (the past 7 days of temperatures) and calculates: “If yesterday was 35°C and the day before was 33°C, tomorrow is likely to be 36°C.”

This technique is called Autoregression, which means “Predicting yourself based on your own past values.”

📊 Chapter 3: Famous Time Series Models

Data Scientists use several different mathematical recipes to predict time series data.

1. ARIMA (AutoRegressive Integrated Moving Average)

  • What it is: ARIMA is one of the oldest and most reliable time series models. It is a math formula that analyzes the correlation between today’s value and past values (AutoRegressive) and the correlation between errors (Moving Average).
  • Where it’s used: Banks use ARIMA to predict stock market prices. Governments use it to predict GDP growth.
  • The catch: ARIMA struggles if the seasonality is very complex. It hates weird patterns.

2. Prophet (By Meta/Facebook)

  • What it is: Prophet was created by Facebook engineers to handle the messy, seasonal data that ARIMA struggled with. It is designed to easily handle daily, weekly, and yearly seasonal patterns—perfect for predicting website traffic or store sales.
  • Where it’s used: Almost every large tech company uses Prophet to forecast server usage, user sign-ups, and ad revenue.
  • The catch: Prophet needs a lot of historical data (usually at least 2 years) to work well. It can’t predict with only 5 days of data.

3. LSTM (Long Short-Term Memory)

  • What it is: We learned about RNNs in the Transformers article—they remember previous inputs. LSTM is a supercharged version of an RNN specifically designed for time series. It has a built-in “Memory Gate” that allows it to remember patterns from 100 days ago while focusing on today’s data.
  • Where it’s used: Scientists use LSTMs to predict the trajectory of hurricanes and to forecast electricity usage for entire cities.
  • The catch: LSTMs require massive, massive amounts of data and expensive supercomputers to train. They are very slow compared to Prophet.

🌍 Chapter 4: Where is Time Series Forecasting Used?

Time Series AI isn’t just for scientists in labs. It is used every day in critical infrastructure.

1. Energy Grids (The Power Forecast)

Electricity cannot be stored efficiently in large amounts. Power plants have to generate exactly the right amount of electricity at the exact moment it is needed. If they make too much, it’s wasted. If they make too little, there is a blackout.

  • Power companies use Time Series AI to look at the past 10 years of electricity usage.
  • They predict: “Next Tuesday at 7:00 PM, everyone will turn on their air conditioners because it will be hot, and because there is a soccer match on TV.”
  • They fire up an extra power generator 1 hour before. The lights stay on, and the city never notices a single flicker.

2. Retail and Supply Chains (Inventory Management)

Supermarkets like FairPrice or Cold Storage need to know exactly how many eggs, milk, and bread to order from the farm.

  • They feed the past 5 years of “Bread Sales” into the Time Series AI.
  • The AI notices that every Friday, bread sales spike by 50% because people buy groceries for the weekend.
  • The AI predicts: “Next Friday, we will sell 5,000 loaves of bread.”
  • They order exactly 5,000 loaves. If the AI is accurate, zero bread is thrown away (saving money), and zero customers leave empty-handed (keeping them happy).

3. Climate Science (Weather Predictions)

Meteorologists feed decades of atmospheric pressure, humidity, and wind speeds into massive supercomputers. The Time Series AI processes the Seasonality (Summer is hot, Winter is cold) and the recent Lags (Yesterday’s thunderstorms). This is how weather apps like AccuWeather know it will rain at 3:00 PM tomorrow in your specific neighborhood.

4. Healthcare (Pandemic Tracking)

During the COVID-19 pandemic, governments used Time Series AI to predict the number of hospital beds needed.

  • They analyzed the past 30 days of infection rates (Trend).
  • They fed the AI the “Lag” data: “If infections rose by 10% yesterday, they will rise by 8% today.”
  • The AI predicted the exact week hospitals would be overwhelmed, allowing them to build emergency tents in advance.

📉 Chapter 5: The Rules of Time Series (Stationarity)

Before a Data Scientist feeds data into ARIMA or Prophet, they have to check if the data is Stationary.

What is Stationarity?

A time series is “Stationary” if its statistical properties (mean, variance) do not change over time.

  • Non-Stationary: A stock price that consistently goes UP over 20 years. The mean (average) keeps shifting higher.
  • Stationary: The daily temperature in Singapore. It swings between 29°C and 35°C year after year, but the average is always around 32°C.

Why it matters

Most Time Series math equations assume the data is stationary. If the data has a huge upward Trend (like stock prices), the AI will get confused and make terrible predictions. It will think a $10 stock will become $10,000 by next year.

The Fix: Differencing

To fix non-stationary data, Data Scientists use a technique called Differencing.

  • Instead of feeding the AI the actual stock price ($100, $105, $110), they feed the AI the differences between the prices: ($+5, +5, +5).
  • By removing the Trend and only looking at the changes, the data becomes stationary, and the model can accurately predict if the stock will go up or down by $5 tomorrow.

💻 Chapter 6: Forecast Error (Why AI Can’t Be 100% Right)

Even the best Time Series AI will occasionally get it wrong. Data Scientists measure this using a metric called Forecast Error.

Why do errors happen?

  1. Unexpected Shocks: An earthquake, a sudden war, or a global pandemic. These are the “Noise” we talked about. The AI has never seen such a massive shock in its historical data, so it cannot predict it.
  2. Regime Changes: Suddenly, a new technology appears that changes human behavior overnight. For example, the smartphone was invented. Old time series models that predicted “People will use the internet on desktop computers forever” suddenly became completely broken because the entire Trend shifted to mobile.

The Two Types of Errors

When an AI predicts a number (e.g., Tomorrow’s temperature is 30°C), and the real number is 28°C, that error matters.

  • Mean Absolute Error (MAE): This just calculates the average distance between the predictions and the real numbers. (Absolute means we ignore the minus sign. A prediction 2°C higher and 2°C lower both count as 2).
  • Root Mean Squared Error (RMSE): This is stricter. It squares the errors before averaging them. A 5°C error is punished much harder than a 2°C error. This forces the AI to avoid huge, massive mistakes at all costs.

💼 Chapter 7: Careers in Time Series Forecasting

1. Supply Chain Forecaster (The Inventory Manager)

  • What they do: They use Time Series AI to predict how many products customers will want to buy in the next 6 months. If they predict too high, the company loses millions in unsold inventory. If they predict too low, customers leave and never come back.
  • Average Salary: $130,000+ USD / year.

2. Quantitative Analyst (The Financial Forecaster)

  • What they do: They are hired by hedge funds and investment banks. They build highly advanced Time Series models (using LSTMs) to predict exactly whether a stock will go up or down in the next 5 minutes. They buy and sell millions of dollars of stock based on their AI’s predictions.
  • Average Salary: $180,000+ USD / year.

3. Energy Trading Analyst (The Power Predictor)

  • What they do: They predict the price of crude oil and natural gas months in advance. They feed past oil prices, global political news, and weather data into the model. If they predict a cold winter in Europe, they know oil prices will spike, and they advise the company to buy oil now.
  • Average Salary: $150,000+ USD / year.

🧪 Chapter 8: Experiment – Forecasting with Prophet in Python

You don’t need to be a quantitative analyst to use Time Series. There is a free, open-source Python library called Prophet that is incredibly easy to use.

The “Ice Cream Sales” Forecast

  1. Make sure you have Python installed. In your terminal, type: pip install prophet pandas matplotlib
  2. Create a file forecast.py and paste the following code. This takes a real-world dataset of 5 years of ice cream sales in a shop.
import pandas as pd
from prophet import Prophet

# Create a fake dataset of ice cream sales over 5 years
data = pd.DataFrame({
    'ds': pd.date_range(start='2019-01-01', periods=365*5, freq='D'),
    'y': [10 + (i % 30) + (i % 365)/5 for i in range(365*5)]
})

# Initialize the Prophet model
model = Prophet()

# Train the model on the data
model.fit(data)

# Create a future dataframe for the next 1 year (365 days)
future = model.make_future_dataframe(periods=365)

# Predict the future!
forecast = model.predict(future)

# Plot the predictions
fig1 = model.plot(forecast)
fig2 = model.plot_components(forecast)

print("Forecast completed! Check the graph file generated.")

🏁 Conclusion: Peering into the Future

Time Series Forecasting is one of the most practical and powerful applications of AI. It helps us plan for the future, avoid disasters, and make better decisions.

We’ve Learned

  • Time Series data is collected over time at regular intervals

  • It has 3 components: Trend, Seasonality, and Noise

  • Lags are used to predict future values

  • Famous models include ARIMA, Prophet, and LSTM

  • It’s used in energy, retail, weather, and healthcare

  • Stationarity is important for accurate predictions

  • Forecast Error measures how accurate predictions are

What This Means for You

Understanding Time Series helps you:

  • Understand how predictions are made

  • Appreciate why AI can’t predict everything

  • See the importance of historical data

In Our Next Article:

Now that you understand Time Series, it’s time to explore Explainable AI (XAI) —opening the black box!

🌳 Back to AI Family Tree