Batches documentation & index

Academy Batches

View the Project on GitHub empathyco/academy-batches

Data Path - Winter 2023

Content Table

Week 1

Our first week on the Data path of the Academy program at Empathy.co was focused on Spark. We received several classes regarding Spark, mainly learning about how to work with it and the several functions that this framework. The following people were the people in charge of delivering said classes.

Path Owner Content Deliverers
Daniel Hernandez Daniel Hernández
  David Casado

Key learning points

  1. Working with spark and scala
  2. Learning about how to manage and query dataframes
  3. Learning about the spark functions
  4. Learning about spark window

Spark exercises

Week one was a week of comming into contact with the framework and programming language that we will be using. We went over Spark and the functions that this framework has as well as some examples of code both in scala and python.

The idea behind this classes was to get acquiantanced with spark and querying over large datasets, getting the information needed about them and displaying them on the console or even saving them locally.

These are some examples made by the Academy Fellows (The repository is part of the company and private so someone not everyone might be able to see it):

Week 2

6 - 12 march

Our second week on the Data path of the Academy program at Empathy.co was focused on databases. We received classes regarding PostgreSQL and MongoDB. We also kept on doing the Spark exercises of the week before. The following people were the ones in charge of delivering said classes.

Path Owner Content Deliverers
Daniel Hernandez Jorge Yagüe París
  David Casado Corona

Key learning points

  1. We kept on working with spark and scala.
  2. Learning about PostgreSQL: setting it up, storing data, and doing some exercises.
  3. Learning about MongoDB: setting it up in our local machine (either by brew or in a docker container), learning how to store and query data, etc.
  4. Do the MongoDB University - Introduction to MongoDB course.
  5. Do the MongoDB University - M121: The MongoDB Aggregation Framework course.

PostgreSQL

Our first class this week was hosted by Jorge and it was about learning how PostgreSQL works, which is an open source object-relational database system used at Motive in Amazon RDS (AWS Redshift).

The idea behind this classes was to get acquiantanced with setting up PostgreSQL in our local machine, connect to it and do some tests (create database, store data, select, insert).

MongoDB

Our second class this week was hosted by David and it was about learning MongoDB. MongoDB is a NoSQL, document-orientended database program, that uses JSON-like documents (BSON). In this class we learned how to set it up in our local machine (either by brew or in a docker container) and we also did some tests (how documents are stored, do some queries, etc). We learned all the basics and some more advanced stuff like the aggregation framework.

To finish up, David explained to us the exercises we have to do for this module.

This week has been primarely about doing both courses, specially the one about the Aggregation Framework, and doing the exercises.

These are some examples made by the Academy Fellows (The repository is part of the company and private so someone not everyone might be able to see it):

Extra

We attended “Sashimi-Tuesday: Workshop on Public Speaking, by Sam Gratton” where we learned about public speaking, and “Internal Areas Review”. Also, like very week, we had our weekly office hour, and our daily academy fellows update.