Lesson 2 of 7 · All lessons in this module
  1. Fintech and the four Vs of Big Data
  2. Sources of Big Data and its challenges
  3. AI and how machine learning learns
  4. Overfitting, underfitting and model fit
  5. Supervised, unsupervised and deep learning
  6. Data science: processing and visualisation
  7. Text analytics, NLP and data science tools

Lesson 2 of 7 · 13 min

Sources of Big Data and its challenges

Alternative data from individuals, business processes and sensors can reveal performance before traditional reports do, but they are messy, can be biased and may raise legal and ethical problems.

In short

  • Traditional data: annual reports, regulatory filings, sales and earnings figures, conference calls, and market prices and volumes.
  • Alternative data have three main sources: individuals (mostly unstructured), business processes (mostly structured, often leading indicators) and sensors (by far the largest volume).
  • The Internet of Things (IoT) is the network of everyday objects fitted with sensors, software and connections that share data.
  • Investors use alternative data to find new price drivers, select assets, improve trade execution and spot trends; specialised vendors now sell such datasets.
  • Challenges: selection bias, missing data, outliers, too little data and data unsuited to the question; data must be sourced, cleansed and organised first.
  • Data not in the public domain, such as web-scraped personal details, raise legal and ethical issues, and rules differ across jurisdictions.

Unlock this lesson free for 7 days

Create a free account to get 7 days free: every lesson, video and flashcard, and the question bank. No card needed.

Start your 7-day free trialLog in