Start here

How to use this siteSearch on YouTubeBookmarkSaved

1. Learn in order. Each module uses the one before it. The order is: what big data is, Hadoop storage and processing, Pig, Hive, Scala and Spark, then streaming.
2. Work every example by hand before you read the solution. The endsem numericals are of these types:
   • HDFS blocks, last-block size, replicas and raw storage (l03).
   • MapReduce word-count traces and partition = hash(key) mod reduce tasks (l04).
   • Pig and Hive outputs: GROUP, JOIN, COGROUP, CROSS, aggregates, bucket numbers, bitmap index vectors (l05–l10).
   • Spark disk passes and MLlib feature values: MinMaxScaler, StandardScaler, StringIndexer, Spearman correlation (l12, l13).
   • GraphX degrees, filters and neighbour sums (l14).
   • Stream windows, watermarks, reservoir sampling, Bloom filters, Kafka partitions and consumer groups (l15, l16).
3. Do the self-test at the bottom of each page.
4. The night before the exam, read only the Recap boxes and the formula cheatsheet.

Press Ctrl K to search every lecture. Each page has Copy Markdown and Open buttons to take the notes into ChatGPT or Claude. Mark a lecture as done to tick it in the sidebar. Bookmark any section from its heading; your bookmarks collect on the Bookmarks page.

The course mapSearch on YouTubeBookmarkSaved