$29
Assignment 4 Solution
Late Submissions are not accepted. A deduction of 10% per day will be applied for late submission.
To avoid any additional charges for resource consumption - Delete any AWS service, storage, database after fulfilling the assignment submission requirements
Objective:
This assignment will help you learn some key services of AWS platform. In this assignment, you are required to work on GCP ML, and AWS Comprehend.
Plagiarism Policy:
• This assignment is an individual task. Collaboration of any type amounts to a violation of the academic integrity policy and will be reported to the AIO.
• Content cannot be copied verbatim from any source(s). Please understand the concept and write in your own words. In addition, cite the actual source. Failing to do so will be considered as plagiarism and/or cheating.
• The Dalhousie Academic Integrity policy applies to all material submitted as part of this course. Please understand the policy, which is available at: https://www.dal.ca/dept/university_secretariat/academic-integrity.html
Assignment Rubric - based on the discussion board rubric (McKinney, 2018)
Excellent (25%)
Proficient (15%)
Marginal (5%)
Unacceptable (0%)
Problem # where
applied
Completeness
All required tasks
Submission
Some tasks are
Incorrect and
Part A
including
are completed
highlights tasks
completed, which
irrelevant
Citation
completion.
are disjoint in
However, missed
nature.
some tasks in
between, which
created a
disconnection
Correctness
All parts of the
Most of the given
Most of the given
Incorrect and
Part C
given tasks are
tasks are correct
tasks are incorrect.
unacceptable
correct
However, some
The submission
portions need minor
requires major
modifications.
modifications.
Novelty
The submission
The submission
The submission
There is no novelty
Part C
contains novel
lacks novel
does not contain
contribution in key
contributions. There
novel
segments, which is
are some evidence of
contributions.
a clear indication
novelty, however, it
However, there is
of application
is not significant
an evidence of
knowledge.
some effort.
Clarity
The written or
The written or
The written or
Failed to prove the
Part B
graphical
graphical materials,
graphical materials,
clarity. Need proper
Fall 2021 saurabh.dey@dal.ca
materials, and
developed
applications
provide a clear
picture of the
concept and
highlights the
clarity.
and developed
applications do not
show clear picture of
the concept. There is
room for
improvement
and developed applications fail to prove the clarity. Background knowledge is needed.
background
knowledge to perform
the tasks.
Citation:
McKinney, B. (2018). The impact of program-wide discussion board grading rubrics on students’ and faculty satisfaction. Online Learning, 22(2), 289-299.
Tasks:
This assignment has 3 parts. Part A is related to background study and report writing. Part B, and C are related to coding, development, and testing
Part A. Read AWS official documentation on SageMaker and Comprehend. Learn how these services are used in event-driven applications and explore various use cases.
Once you learn about the services, write ½ page summary on each service highlighting the usage, and a statement on how you will be using these services for “car rental application” or “home security application” (You can pick any one application).
Please ensure that you have included the proper citation. Do not copy paste any content from the source documentations. The summary must be written in your own words.
Part A - Submission requirement:
A pdf file with the summary, your approach, and citation.
Part B. Build an event-driven serverless application using GCP ML.
In this part of the assignment, you need to use GCP file storage, Cloud function. [B00xxxxxx = your B00 number] used in bucket/GCP storage naming
take screenshots at every step and submit as part of the PDF:
a. Create your 1st storage bucket SourceDataB00xxxxxx and upload the files (from 001 to 299) given in the Train folder. You need to write a script or use the SDK to upload the files on the bucket.
b. Once a file is uploaded, a cloud function - “generateVector” should extract words from all the files (remove the stop words). Then compute Levenshtein distance(https://en.wikipedia.org/wiki/Levenshtein_distance) between the Current, and Next word. Consider the sentence “Asia is a big continent”
Fall 2021 saurabh.dey@dal.ca
E.g. If “Asia” is the current word, and “big” is the next word, then Levenshtein distance (or edit distance) will be “3”. (remove the stop words)
e.g. Contents of “trainVector.csv” should be
Current_Word
Next_Word Levenshtein_distance
Asia
big
4
big
continent
8
c. This file (“trainVector.csv”) is saved in a new bucket TrainDataB00xxxxxx
d. GCP ML should get the training data for a clustering algorithm (KMenas) from the TrainDataB00xxxxxx bucket.
e. Once the training is done, like point (a), upload the test files given in the Test folder (300 to 401) to SourceDataB00xxxxxx
f. The cloud function generateVector computes the distance vector same as point
(b) and store it in “testVector.csv”.
g. This file (testVector.csv) is saved in another new bucket, TestDataB00xxxxxx
h. GCP ML should get the test data for KMeans algorithm from the TestDataB00xxxxxx bucket.
i. Finally, write a code or configure a service to obtain information about clusters (e.g. centroids, cluster numbers, outliers etc. which are generated by GCP ML), and display the clusters.
j. Test your functions, and entire cluster generation process.
Part B - Submission requirement:
Submit screenshots of every steps. Please do not exclude any steps. Include all screenshots as part of a PDF file. In addition, provide the program/scripts as part of the PDF file. In addition, submit code in gitlab.
Hint: If needed, you can use libraries for Levenshtein distance calculation. However, add citation.
Part C. Build an event-driven serverless application using AWS Comprehend.
In this part of the assignment, you need to use S3 bucket, Lambda Functions, and AWS Comprehend.
[B00xxxxxx = your B00 number] used in bucket naming
take screenshots at every step and submit as part of the PDF:
a. Create your 1st S3 bucket TwitterDataB00xxxxxx and upload the given tweets file. You need to write a script or use the SDK to upload the files on the bucket.
b. To perform any pre or post processing of the files, you can write Lambda functions.
Fall 2021 saurabh.dey@dal.ca
c. Once the file containing all the tweets is uploaded on the bucket, AWS Comprehend is used to perform sentiment analysis of tweets.
d. Your output should be captured in a csv or json format.
Data Source Acknowledgement: The tweets file was generated by Alapati Lakshmi Manjari, 2019.
Part C - Submission requirement:
Submit screenshots of every steps. Please do not exclude any steps. Include all screenshots as part of a PDF file. In addition, provide the program/scripts as part of the PDF file. In addition, submit code in gitlab.
Fall 2021 saurabh.dey@dal.ca
