Specifications

Your task is to develop a Lambda function that is called once a new JSON object is created in an S3 bucket. Once the Lambda function is called, it will read the JSON objects, parse them and insert them into a DynamoDB table. Your task also includes preparing a dataset by applying pre-processing techniques to it. Your submission will be evaluated based on:

• whether instructions have been followed;

• correct functionality;

• correct implementation; and

• comments inside the program

Overview: The assignment has two parts. In the first part, you have to implement a Lambda function that is connected with an S3 bucket by a trigger. The trigger should allow the Lambda function to get notified for all ‘Object Created’ events. This means that once a new object is created into that bucket, your Lambda function will be invoked. As a part of the invocation, your Lambda function should receive the event and bucket details. Your Lambda function will use this information to read the content of the object and insert some relevant entries into a DynamoDB table.

In the second part, you will prepare a given dataset by applying several data pre-processing methods to it.

Details: The details of the assignment tasks are given below.

Part A: Lambda function and handler app: Implement the handler app in Cloud9 and create a Lambda function that gets notified once an object (i.e., employee.json, attached with the assignment) is created in the specified S3 bucket. The Lambda function then needs to read the content of the object, parse the information, and insert the parsed information into the below DynamoDB table. Employee (Id, Name, Make, Age, Role); Name, Role: String, and Id, Age: Number

You need to use a specific AWS IAM role (robomaker_students) while creating your Lambda function. That particular role and its associated policy (Probomaker_students) do not allow DynamoDB access by default. Therefore, to make your Lambda function and tester app work, you need to make some changes to this policy.

Testing the Lambda function: You can test the Lambda function by just simply uploading the employee.json into the specified bucket.

Requirements for Part A: Your website and Cloud9 app must fulfill the following requirements. You will lose marks otherwise.

• Your Cloud9 app must be named yourscuusernameA2App.

• You will use a single bucket for the whole assignment and its name must be yourscuusername-a2-bucket.

• Your Cloud9 (handler) app must use exception handling and loop(s).

• You must use AWS SDK v 1.x to implement the Cloud9 app.

• You must implement the assignment in the SCU-provided AWS account, personal AWS account will not be accepted.

• You must use the robomaker_students role and its associated policy.

• You need to add comments inside the source code.

Solution Hints: Check the following hints for implementing Part A.

• You need to use the “org.json” package to retrieve the JSON array and parse each JSON object. Please check this link: https://stleary.github.io/JSON-java/index.html. You can use the latest maven dependency from here: https://mvnrepository.com/artifact/org.json/json.

• A JSONArray is a collection of JSONObjects and a JSONObject is a collection of values of each data filed. You need to understand this concept and parse the fields.

Part B: Data preparation pipeline: In the second part, you need to work on a dataset resolution.csv (attached with this assignment). This dataset comes from the AWS repository. You need to create, publish, and import a recipe to complete the following tasks in the Glue DataBrew project.

• Replacing all missing values in any two columns which have missing values. You can apply any method to compute the new value.

• Detect which rows of the dataset are duplicate rows.

• Compute the total number of “no votes” for each distinct “vote date” and add a new column to the dataset.

• Compute the total number of “yes votes” for each distinct “vote date” and add a new column to the dataset.

resolution.csv