Functional Programming🔗
This section of the course is not given this year.
Functional Programming for Distributed Data🔗
Introduction to Julia🔗
As the first exercise, you'll need to install Julia and IJulia locally or make a working Julia Colab Notebook. While Colab is sufficient for today's exercises, it is recommended to make a local installation:
Julia download Julia kernel for Jupyter
Here is a Colab template from this Github repository which will install the Julia kernel for a single Colab instance.
Once you have a Julia Jupyter kernel, follow this Julia for Pythonistas notebook.
Functional Programming in Julia🔗
Julia documentation explaining:
- Functions, showing that they are first-class
- the
mapfunction which is a higher-order function - distributed computing allowing for transfer of functions between threads or workers
Distributed Data in Julia🔗
Julia's base language supports distributed calculation but there are a few packages which facilitate data processing tasks over distributed data:
- DistributedArrays - A general Array type which can be distributed over multiple workers.
- JuliaDB - A data structuring package which automatically handles distributed data storage and computation
- Spark.jl - A Julia interface to Apache Spark. Related blog post.
Map Reduce Exercise🔗
The second part of this class is an interactive notebook in the Julia language covering the MapReduce programming framework, from simple addition queries to a grep example.
MapReduce notebook on Colab (requires adding Julia kernel installation)