Functional Programming

This section of the course is not given this year.

Functional Programming for Distributed Data

Link to slides

Introduction to Julia

As the first exercise, you'll need to install Julia and IJulia locally or make a working Julia Colab Notebook. While Colab is sufficient for today's exercises, it is recommended to make a local installation:

Julia download Julia kernel for Jupyter

Here is a Colab template from this Github repository which will install the Julia kernel for a single Colab instance.

Once you have a Julia Jupyter kernel, follow this Julia for Pythonistas notebook.

Github Colab

Functional Programming in Julia

Julia documentation explaining:

• Functions, showing that they are first-class
• the map function which is a higher-order function
• distributed computing allowing for transfer of functions between threads or workers

Distributed Data in Julia

Julia's base language supports distributed calculation but there are a few packages which facilitate data processing tasks over distributed data:

• DistributedArrays - A general Array type which can be distributed over multiple workers.
• JuliaDB - A data structuring package which automatically handles distributed data storage and computation
• Spark.jl - A Julia interface to Apache Spark. Related blog post.

Map Reduce Exercise

The second part of this class is an interactive notebook in the Julia language covering the MapReduce programming framework, from simple addition queries to a grep example.

MapReduce notebook

MapReduce notebook on Colab (requires adding Julia kernel installation)