Starting from:
$30
EECS485-Project 2 Server-side Dynamic Pages Solved
An Instagram clone implemented with server-side dynamic pages. This is the second of an EECS 485 three project sequence: a static site generator from templates, server-side dynamic pages, and client-side dynamic pages.
Build an interactive website using server-side dynamic pages. Reuse the templates from project 1, rendering them on-demand when a user loads a page. New features include creating, updating, and deleting users, posts, comments, and likes.
The learning goals of this project include server-side dynamic pages, CRUD (Create, Read, Update, Delete), sessions, and basic SQL database usage.
Here’s a preview of what your finished project will look like. A database-backed interactive website will work (mostly) like the real Instagram.
$ ./bin/insta485run
* Serving Flask app "insta485"
* Running on http://127.0.0.1:8000/ (Press CTRL+C to quit)
Then you will navigate to http://localhost:8000 and see working, multi-user, interactive website that you created.
This project adds lots features. For example, users can add likes and comments.
This spec will walk you through several parts:
1. Setup
2. Building the database
3. Creating starter files
4. Dynamic Server-side Insta485 specification
5. Deploy to AWS
6. Submitting and grading
7. FAQ
Use the same local development tool chain for this project that you did in the previous project.
Setup
Group registration
Register your groups registered on the Autograder. The office hours queue will give first priority to groups asking a question for the first time in a day.
AWS account and instance
You will use Amazon Web Services (AWS) to deploy your project. AWS account setup may take up to 24 hours, so get started now. Create an account, launch and configure the instance. Don’t deploy yet. AWS Tutorial.
Project folder
Create a folder for this project (instructions). Your folder location might be different.
$ pwd
/Users/awdeorio/src/eecs485/p2-insta485-serverside
Version control
Set up version control using the Version control tutorial.
Be sure to check out the Version control for a team tutorial.
After you’re done, you should have a local repository with a “clean” status and your local repository should be connected to a remote GitLab repository.
$ pwd
/Users/awdeorio/src/eecs485/p2-insta485-serverside
$ git status
On branch master
Your branch is up-to-date with 'origin/master'.
nothing to commit, working tree clean
$ git remote -v
origin https://gitlab.eecs.umich.edu/awdeorio/p2-insta485-serverside.git (fetch) origin https://gitlab.eecs.umich.edu/awdeorio/p2-insta485-serverside.git (push) You should have a .gitignore file (instructions).
$ pwd
/Users/awdeorio/src/eecs485/p2-insta485-serverside
$ head .gitignore
This is a sample .gitignore file that's useful for EECS 485 projects.
...
Python virtual environment
Create a Python virtual environment using the Project 1 Python Virtual Environment Tutorial.
You should now have Python tools installed locally.
$ pwd
/Users/awdeorio/src/eecs485/p2-insta485-serverside
$ ls env
$ source env/bin/activate
$ which python
/Users/awdeorio/src/eecs485/p2-insta485-serverside/env/bin/python
$ which pip
/Users/awdeorio/src/eecs485/p2-insta485-serverside/env/bin/pip
Install utilities
Linux and Windows 10 Subsystem for Linux
$ sudo apt-get install sqlite3 curl
MacOS
$ brew install sqlite3 curl
Starter files
Download and unpack the starter files.
$ pwd
/Users/awdeorio/src/eecs485/p2-insta485-serverside
$ wget https://eecs485staff.github.io/p2-insta485-serverside/starter_files.tar.gz $ tar -xvzf starter_files.tar.gz
Move the starter files to your project directory and remove the original starter_files/ directory.
$ pwd
/Users/awdeorio/src/eecs485/p2-insta485-serverside
$ mv starter_files/* .
$ rm -rf starter_files starter_files.tar.gz
You should see these files.
$ tree
.
├── VERSION
├── bin
│ └── insta485test-html
├── setup.py
├── sql
│ └── uploads
...
│ └── e1a7c5c32973862ee15173b0259e3efdb6a391af.jpg └── tests
....
└── util.py
VERSION
Version of the starter files
bin/insta485test-html
Script to test HTML5 compliance
setup.py
Insta485 python package configuration
sql/uploads/
Sample image uploads
tests/
Public unit tests
Before making any changes to the clean starter files, it’s a good idea to make a commit to your Git repository.
Database
If you’re new to SQL, take a look at the w3Schools SQL Intro.
Complete the SQLite Tutorial. After the tutorial, you should have the sqlite3 command line utility installed. Your version might be different.
$ sqlite3 --version
3.29.0 2019-07-10 17:32:03 fc82b73eaac8b36950e527f12c4b5dc1e147e6f4ad2217ae43ad82882a88bfa
You should have a script to manage the database.
$ pwd
/Users/awdeorio/src/eecs485/p2-insta485-serverside
$ ./bin/insta485db reset
+ rm -rf var/insta485.sqlite3 var/uploads
+ mkdir -p var/uploads
+ sqlite3 var/insta485.sqlite3 < sql/schema.sql
+ sqlite3 var/insta485.sqlite3 < sql/data.sql
+ cp sql/uploads/* var/uploads/ You have created these files.
$ pwd
/Users/awdeorio/src/eecs485/p2-insta485-serverside
$ tree sql var sql ├── data.sql
├── schema.sql └── uploads
...
└── e1a7c5c32973862ee15173b0259e3efdb6a391af.jpg var
├── insta485.sqlite3 └── uploads
...
└── e1a7c5c32973862ee15173b0259e3efdb6a391af.jpg
Schema
Update schema.sql , which will create 5 tables: users , posts , following , comments and likes . The following list describes the tables and columns
users table username , at most 20 chars, primary key fullname , at most 40 chars email , at most 40 chars filename , at most 64 chars password , at most 256 chars, created , TIMESTAMP type, automatically set by SQL engine to current date/time.
posts table postid , integer, primary key filename , at most 64 chars owner , at most 20 chars, foreign key to users created , TIMESTAMP type, automatically set by SQL engine to current date/time. NOTE: rows are automatically removed and updated (read about CASCADE ).
following table username1 , at most 20 chars, foreign key to users username2 , at most 20 chars, foreign key to users
The tuple ( username1 , username2 ) form a primary key created , TIMESTAMP type, automatically set by SQL engine to current date/time.
NOTE: rows are automatically removed and updated (read about CASCADE ).
comments table commentid , integer, primary key owner , at most 20 chars, foreign key to users table postid , integer, foreign key to posts table text , at most 1024 chars created , TIMESTAMP type, automatically set by SQL engine to current date/time. NOTE: rows are automatically removed and updated (read about CASCADE ).
likes table owner , at most 20 chars, foreign key to users postid , integer, foreign key to posts created , TIMESTAMP type, automatically set by SQL engine to current date/time.
The tuple ( owner , postid ) form a primary key
NOTE: rows are automatically removed and updated (read about CASCADE ).
Data
Update sql/data.sql to add all initial data. You can find a complete dump of the initial data in insta485db-dump.txt. Your timestamps will be different. The passwords are all set to password .
Testing
Run the public autograder testcases on your database schema and data. You will need to complete the Flask Tutorial before running this test.
$ pytest -v tests/test_database_public.py
...
========================== 2 passed in 1.76 seconds ===========================
Make sure these tests pass before moving on. The other unit tests rely on a fully functionally bin/insta485db script.
Flask app
Complete the Flask Tutorial.
You should now have a directory containing an insta485 Python module.
$ tree insta485 insta485 ├── __init__.py ├── config.py
├── model.py
├── static
│ └── css
│ │ └── style.css
├── templates
│ └── index.html
└── views
├── __init__.py
└── index.py
The insta485run script starts a development server and you can browse to http://localhost:8000/ where you’ll see your “hello world” app.
$ ./bin/insta485run
+ test -e var/insta485.sqlite3
+ export FLASK_DEBUG=True
+ FLASK_DEBUG=True
+ export FLASK_APP=insta485
+ FLASK_APP=insta485
+ export INSTA485_SETTINGS=config.py
+ INSTA485_SETTINGS=config.py
+ flask run --host 0.0.0.0 --port 8000
* Serving Flask app "insta485" (lazy loading)
* Environment: production
WARNING: Do not use the development server in a production environment. Use a production WSGI server instead.
* Debug mode: on
* Running on http://0.0.0.0:8000/ (Press CTRL+C to quit)
Testing
Compliant HTML
Automatically generated HTML shall be W3C HTML5 compliant. To test dynamically generated pages, we’re going crawl the site using wget to download each page. Then, we’ll validate the pages using html5validator just like we did in project 1. Starting a second process in a script gets a little complicated, so we’ve provided a script in the starter files, bin/insta485test-html .
NOTE: bin/insta485test-html uses your bin/insta485run , so make sure that works correctly.
NOTE: bin/insta485test-html uses wget which sends HEAD requests, not GET requests.
NOTE: You will not pass this test until the /accounts/logout has been implemented.
$ ./bin/insta485test-html
...
========================================
./bin/insta485test-html
PASS
========================================
insta485test script
Similar to project 1, all Python code must be PEP8 compliant, comments shall be PEP257 compliant, and code shall pass a pylint static analysis.
Write another script called bin/insta485test that does this:
1. Stop on errors and prints commands
2. Run pycodestyle insta485
3. Run pydocstyle insta485
4. Run pylint --disable=cyclic-import insta485
5. Run all unit tests using pytest -v tests
6. Run the provided bin/insta485test-html script to validate your app’s html, as described above
Don’t forget to check for shell script pitfalls.
$ file bin/*
bin/insta485db: Bourne-Again shell script text executable, ASCII text bin/insta485run: Bourne-Again shell script text executable, ASCII text bin/insta485test: Bourne-Again shell script text executable, ASCII text bin/insta485test-html: Bourne-Again shell script text executable, ASCII text
Unit tests
Now that you have the framework of the project in place and your utility scripts written, we can run some of the unit tests. We have provided the public Autograder tests.
Run the public autograder testcases on your utility scripts.
$ pytest -v tests/test_scripts_public.py
...
========================== 6 passed in 1.16 seconds ===========================
Note: if you get deprecation warnings from third party libraries, check out the pytest tutorial deprecation warnings to suppress them.
Dynamic Server-side Insta485 specification
This project includes the same pages as project 1. The pages also include buttons to follow, unfollow, like, unlike and comment. We’ll also add pages for user account administration.
URLs
List of URLs from project 1, including screenshots with user awdeorio logged in.
/ screenshot
/u/<user_url_slug/ screenshot
/u/<user_url_slug/followers/ screenshot
/u/<user_url_slug/following/ screenshot
/p/<postid_slug/ screenshot
/explore/ screenshot List of new URLs:
/accounts/login/ screenshot (no user logged in)
/accounts/logout/ Immediately redirects to /accounts/login/ . No screenshot.
/accounts/create/ screenshot (no user logged in)
/accounts/delete/ screenshot
/accounts/edit/ screenshot
/accounts/password/ screenshot
Hint: When linking to pages or static files look into flask’s url_for() and send_from_directory() function.
All pages
Include a link to the main page / .
If not logged in, redirect to /accounts/login/ .
If logged in, include a link to /explore/ .
If logged in, include a link to /u/<user_url_slug/ where user_url_slug is the logged in user.
Access control
The server should accept POST requests from the logged in user only. To reject a request with a permissions error, use flask.abort(403) .
The following examples assume you have a (mostly) working Insta485 project with a freshly reset database.
Normal example
This example is just like using a web browser to fill in the HTML login form rendered on the login page and then navigating to / . Then, it deletes a post.
Use curl to sign in to Insta485. This command issues a POST request to /accounts/login/ with a username and password.
$ curl -X POST http://localhost:8000/accounts/login/ \
-F username=awdeorio \
-F password=password \
--cookie-jar cookies.txt
View the index / page by issuing a GET request.
$ curl -b cookies.txt http://localhost:8000/
... <index.html page content here
Delete an Insta485 post. The user awdeorio can successfully delete his own post by issuing a POST request.
$ curl -X POST http://localhost:8000/p/1/ \
-F postid=1 \
-F 'delete=delete this post' \ -b cookies.txt
Malicious example
Even though the “Delete post” button is hidden on posts that the logged in user doesn’t own, any user can use a tool like curl to send a POST request to try to delete an Insta485 post.
Try to delete a post created by jflinn using awdeorio ’s cookies. We get a 403 Forbidden error. This is a good thing!
$ curl -X POST http://localhost:8000/p/2/ \
-F postid=2 \
-F 'delete=delete this post' \ -b cookies.txt
...
<title403 Forbidden</title ...
Index /
screenshot
The index page should include all posts from the logged in user and all other users that the logged in user is following. The most recent post should be at the top. For each post:
Link to the post detail page /p/<postid_slug/ by clicking on the timestamp.
Link to the owner’s page /u/<user_url_slug/ by clicking on their username or profile picture.
Time since the post was created in human-readable format
Number of likes, using correct English
Comments, with owner’s username, oldest at the top
Link to the comment owner’s page /u/<user_url_slug/ by clicking on their username.
“like” or “unlike” button, pick the logical one Comment input and submission button
Form for “like” button
<!-- DO NOT CHANGE THIS (aside from where we say 'FIXME') -- <form action="" method="post" enctype="multipart/form-data"
<input type="hidden" name="postid" value="<FIXME postid HERE"/
<input type="submit" name="like" value="like"/ </form
Form for “unlike” button
<!-- DO NOT CHANGE THIS (aside from where we say 'FIXME') -- <form action="" method="post" enctype="multipart/form-data"
<input type="hidden" name="postid" value="<FIXME postid HERE"/
<input type="submit" name="unlike" value="unlike"/ </form
Form for “comment” button
<!-- DO NOT CHANGE THIS (aside from where we say 'FIXME') -- <form action="" method="post" enctype="multipart/form-data"
<input type="hidden" name="postid" value="<FIXME postid HERE"/
<input type="text" name="text"/
<input type="submit" name="comment" value="comment"/ </form
/u/<user_url_slug/ screenshot
Be sure to include username ( user_url_slug ) Relationship
“following” if the logged in user is following user_url_slug . Also include an “unfollow” button.
“not following” if the logged in user is not following user_url_slug . Also include a “follow” button.
Blank if logged in user == user_url_slug
Number of posts, with correct English
Number of followers, with correct English
Link to /u/<user_url_slug/followers/ Number following
Link to /u/<user_url_slug/following/
Name
A small image for each post
Clicking on the image links to /p/<postid_url_slug/ For a user’s own page, also include
Link to /accounts/edit/
Link to /accounts/logout/
File upload form for creating a new post
Note: creating a new post should not redirect the user
Form for follow button
<!-- DO NOT CHANGE THIS (aside from where we say 'FIXME') -- <form action="" method="post" enctype="multipart/form-data"
<input type="submit" name="follow" value="follow"/
<input type="hidden" name="username" value="<FIXME username"/ </form
Form for unfollow button
<!-- DO NOT CHANGE THIS (aside from where we say 'FIXME') -- <form action="" method="post" enctype="multipart/form-data"
<input type="submit" name="unfollow" value="unfollow"/
<input type="hidden" name="username" value="<FIXME username"/ </form
Form for file upload
<!-- DO NOT CHANGE THIS --
<form action="" method="post" enctype="multipart/form-data"
<input type="file" name="file"
<input type="submit" name="create_post" value="upload new post"/ </form
We use the sha256 hash algorithm to compute a hash of file contents. You can do this at the command line like this:
$ wget http://andrewdeorio.com/assets/headshot.jpg -O awdeorio.jpg
$ sha256sum awdeorio.jpg
e1a7c5c32973862ee15173b0259e3efdb6a391af awdeorio.jpg Here’s how to compute filenames in your Flask app:
import os import shutil import tempfile import flask
def sha256sum(filename):
"""Return sha256 hash of file content, similar to UNIX sha256sum.""" content = open(filename, 'rb').read() sha256_obj = hashlib.sha256(content) return sha256_obj.hexdigest()
# Save POST request's file object to a temp file dummy, temp_filename = tempfile.mkstemp() file = flask.request.files["file"] file.save(temp_filename)
# Compute filename
hash_txt = sha256sum(temp_filename) dummy, suffix = os.path.splitext(file.filename) hash_filename_basename = hash_txt + suffix hash_filename = os.path.join( insta485.app.config["UPLOAD_FOLDER"], hash_filename_basename
)
# Move temp file to permanent location shutil.move(temp_filename, hash_filename)
/u/<user_url_slug/followers/ screenshot
List the users that are following user_url_slug . For each, include:
Icon
Username, with link to /u/<username/
Relationship to logged in user
“following” if logged in user is following username. Also, an “unfollow” button. See above for HTML form.
“not following” if logged in user is not following username. Also, a “follow” button. See above for HTML form.
Blank if logged in user == username
/u/<user_url_slug/following/ screenshot
List the users that user_url_slug is following. For each, include:
Icon
Username, with link to /u/<username/ Relationship to logged in user
“following” if logged in user is following username. Also, an “unfollow” button. See above for HTML form.
“not following” if logged in user is not following username. Also, a “follow” button. See above for HTML form.
Blank if logged in user == username
/p/<postid_slug/
screenshot
This page shows one post. Include the same information for this one post as is shown on the main page / .
Include a “delete” button next to each comment owned by the logged in user.
<!-- DO NOT CHANGE THIS (aside from where we say 'FIXME') -- <form action="" method="post" enctype="multipart/form-data"
<input type="hidden" name="commentid" value="<FIXME commentid"/
<input type="submit" name="uncomment" value="delete"/ </form
Include a “delete this post” button if the post is owned by the logged in user. Delete the file from the filesystem. Delete everything in the database related to this post. Redirect to /u/<user_url_slug/ for the user whose post was deleted.
<!-- DO NOT CHANGE THIS (aside from where we say 'FIXME') -- <form action="" method="post" enctype="multipart/form-data"
<input type="hidden" name="postid" value="<FIXME postid"/
<input type="submit" name="delete" value="delete this post"/ </form
/explore/
screenshot
This page lists all users not that the logged in user is not following and includes:
Icon
Username with link to /u/<user_url_slug/
“follow” button
See above for HTML form
/accounts/login/ screenshot
If logged in, redirect to / .
Otherwise, include username and password inputs, and a login button. If someone tries to log in to a user that does not exist or provides incorrect credentials, abort(403) .
Also include a link to /accounts/create/ in the page.
Use this HTML form code. Feel free to style it and include placeholder s.
<!-- DO NOT CHANGE THIS (aside from styling) --
<form action="" method="post" enctype="multipart/form-data"
<input type="text" name="username"/
<input type="password" name="password"/
<input type="submit" value="login"/
</form
Warning: only store minimal information in a session cookie!
/accounts/logout/
Log out user on the server side. Immediately redirect to /accounts/login/ , which is why this is no screenshot.
/accounts/create/
screenshot
If a user is already logged in, redirect to /accounts/edit/ .
If a user tries to create an account with an existing username in the database, abort(409) - this is an HTTP code indicating a Conflict Error.
If a user tries to create an account with an empty string as the password, abort(400) - this is an HTTP code indicating a Bad Request.
If the account is created successfully, log the user in, and redirect to the main page / .
Also include a link to /accounts/login/ in the page.
HTML form. Style as you like.
<!-- DO NOT CHANGE THIS (aside from styling) --
<form action="" method="post" enctype="multipart/form-data"
<input type="file" name="file"
<input type="text" name="fullname"/
<input type="text" name="username"/
<input type="text" name="email"/
<input type="password" name="password"/
<input type="submit" name="signup" value="sign up"/ </form
See above for file upload and naming procedure.
A password entry in the database contains the algorithm, salt and password hash separated by $ . Use the sha512 algorithm like this:
import uuid import hashlib algorithm = 'sha512' salt = uuid.uuid4().hex hash_obj = hashlib.new(algorithm) password_salted = salt + password
hash_obj.update(password_salted.encode('utf-8')) password_hash = hash_obj.hexdigest() password_db_string = "$".join([algorithm, salt, password_hash]) print(password_db_string)
We’ve shipped one of the autograder tests so you can verify that login and logout mechanics work correctly. It’s in starter_files/tests/test_login_logout_public.py . Save it to
tests/test_login_logout_public.py and run it like this:
$ pytest -v tests/test_login_logout_public.py == test session starts ==
...
== 4 passed in 0.91 seconds ==
/accounts/delete/
screenshot
Confirmation page includes username and this form:
<!-- DO NOT CHANGE THIS --
<form action="" method="post" enctype="multipart/form-data"
<input type="submit" name="delete" value="confirm delete account"/ </form
Delete all post files created by this user. Delete user icon file. Delete all related entries in all tables. Hint: database tables set up properly with primary/foreign key relationships and CASCADE ON DELETE will do this automatically.
Upon successful submission, redirect to /accounts/create/ .
/accounts/edit/
screenshot
Include user’s icon and username, which are not editable.
Include a form with photo upload, name and email. Name and email are automatically filled in with previous value. If no photo file is included, then the server will update only the user’s name and email.
If a photo file is included, then the server will update the user’s photo, name and email. Delete the old photo from the filesystem. See above for file upload and naming procedure.
Link to /accounts/password/ .
Link to /accounts/delete/ .
Use this form
<!-- DO NOT CHANGE THIS (aside from where we say 'FIXME') -- <form action="" method="post" enctype="multipart/form-data"
<input type="file" name="file"
<input type="text" name="fullname" value="<FIXME full name here"/
<input type="text" name="email" value="<FIXME email here"/
<input type="submit" name="update" value="submit"/ </form
Upon successful submission, re-render the current page.
/accounts/password/
screenshot
Include this form:
<!-- DO NOT CHANGE THIS --
<form action="" method="post" enctype="multipart/form-data"
<input type="password" name="password"/
<input type="password" name="new_password1"/
<input type="password" name="new_password2"/ <input type="submit" name="update_password" value="submit"/ </form
Link to /accounts/edit/ .
Check the user’s password and abort(403) if it fails.
Check if both new passwords match. abort(401) otherwise.
Update hashed password entry in database. (See above).
Upon successful submission, redirect to /accounts/edit/ .
Static File Permissions
A user with the direct link to an uploaded file, /uploads/<filename , should only be able to see that file if logged in. If an unauthenticated user attempts to access an uploaded file, abort(403) .
Testing
Run the unit tests. Everything except for the deploy test should pass.
$ pytest -v
Build an interactive website using server-side dynamic pages. Reuse the templates from project 1, rendering them on-demand when a user loads a page. New features include creating, updating, and deleting users, posts, comments, and likes.
The learning goals of this project include server-side dynamic pages, CRUD (Create, Read, Update, Delete), sessions, and basic SQL database usage.
Here’s a preview of what your finished project will look like. A database-backed interactive website will work (mostly) like the real Instagram.
$ ./bin/insta485run
* Serving Flask app "insta485"
* Running on http://127.0.0.1:8000/ (Press CTRL+C to quit)
Then you will navigate to http://localhost:8000 and see working, multi-user, interactive website that you created.
This project adds lots features. For example, users can add likes and comments.
This spec will walk you through several parts:
1. Setup
2. Building the database
3. Creating starter files
4. Dynamic Server-side Insta485 specification
5. Deploy to AWS
6. Submitting and grading
7. FAQ
Use the same local development tool chain for this project that you did in the previous project.
Setup
Group registration
Register your groups registered on the Autograder. The office hours queue will give first priority to groups asking a question for the first time in a day.
AWS account and instance
You will use Amazon Web Services (AWS) to deploy your project. AWS account setup may take up to 24 hours, so get started now. Create an account, launch and configure the instance. Don’t deploy yet. AWS Tutorial.
Project folder
Create a folder for this project (instructions). Your folder location might be different.
$ pwd
/Users/awdeorio/src/eecs485/p2-insta485-serverside
Version control
Set up version control using the Version control tutorial.
Be sure to check out the Version control for a team tutorial.
After you’re done, you should have a local repository with a “clean” status and your local repository should be connected to a remote GitLab repository.
$ pwd
/Users/awdeorio/src/eecs485/p2-insta485-serverside
$ git status
On branch master
Your branch is up-to-date with 'origin/master'.
nothing to commit, working tree clean
$ git remote -v
origin https://gitlab.eecs.umich.edu/awdeorio/p2-insta485-serverside.git (fetch) origin https://gitlab.eecs.umich.edu/awdeorio/p2-insta485-serverside.git (push) You should have a .gitignore file (instructions).
$ pwd
/Users/awdeorio/src/eecs485/p2-insta485-serverside
$ head .gitignore
This is a sample .gitignore file that's useful for EECS 485 projects.
...
Python virtual environment
Create a Python virtual environment using the Project 1 Python Virtual Environment Tutorial.
You should now have Python tools installed locally.
$ pwd
/Users/awdeorio/src/eecs485/p2-insta485-serverside
$ ls env
$ source env/bin/activate
$ which python
/Users/awdeorio/src/eecs485/p2-insta485-serverside/env/bin/python
$ which pip
/Users/awdeorio/src/eecs485/p2-insta485-serverside/env/bin/pip
Install utilities
Linux and Windows 10 Subsystem for Linux
$ sudo apt-get install sqlite3 curl
MacOS
$ brew install sqlite3 curl
Starter files
Download and unpack the starter files.
$ pwd
/Users/awdeorio/src/eecs485/p2-insta485-serverside
$ wget https://eecs485staff.github.io/p2-insta485-serverside/starter_files.tar.gz $ tar -xvzf starter_files.tar.gz
Move the starter files to your project directory and remove the original starter_files/ directory.
$ pwd
/Users/awdeorio/src/eecs485/p2-insta485-serverside
$ mv starter_files/* .
$ rm -rf starter_files starter_files.tar.gz
You should see these files.
$ tree
.
├── VERSION
├── bin
│ └── insta485test-html
├── setup.py
├── sql
│ └── uploads
...
│ └── e1a7c5c32973862ee15173b0259e3efdb6a391af.jpg └── tests
....
└── util.py
VERSION
Version of the starter files
bin/insta485test-html
Script to test HTML5 compliance
setup.py
Insta485 python package configuration
sql/uploads/
Sample image uploads
tests/
Public unit tests
Before making any changes to the clean starter files, it’s a good idea to make a commit to your Git repository.
Database
If you’re new to SQL, take a look at the w3Schools SQL Intro.
Complete the SQLite Tutorial. After the tutorial, you should have the sqlite3 command line utility installed. Your version might be different.
$ sqlite3 --version
3.29.0 2019-07-10 17:32:03 fc82b73eaac8b36950e527f12c4b5dc1e147e6f4ad2217ae43ad82882a88bfa
You should have a script to manage the database.
$ pwd
/Users/awdeorio/src/eecs485/p2-insta485-serverside
$ ./bin/insta485db reset
+ rm -rf var/insta485.sqlite3 var/uploads
+ mkdir -p var/uploads
+ sqlite3 var/insta485.sqlite3 < sql/schema.sql
+ sqlite3 var/insta485.sqlite3 < sql/data.sql
+ cp sql/uploads/* var/uploads/ You have created these files.
$ pwd
/Users/awdeorio/src/eecs485/p2-insta485-serverside
$ tree sql var sql ├── data.sql
├── schema.sql └── uploads
...
└── e1a7c5c32973862ee15173b0259e3efdb6a391af.jpg var
├── insta485.sqlite3 └── uploads
...
└── e1a7c5c32973862ee15173b0259e3efdb6a391af.jpg
Schema
Update schema.sql , which will create 5 tables: users , posts , following , comments and likes . The following list describes the tables and columns
users table username , at most 20 chars, primary key fullname , at most 40 chars email , at most 40 chars filename , at most 64 chars password , at most 256 chars, created , TIMESTAMP type, automatically set by SQL engine to current date/time.
posts table postid , integer, primary key filename , at most 64 chars owner , at most 20 chars, foreign key to users created , TIMESTAMP type, automatically set by SQL engine to current date/time. NOTE: rows are automatically removed and updated (read about CASCADE ).
following table username1 , at most 20 chars, foreign key to users username2 , at most 20 chars, foreign key to users
The tuple ( username1 , username2 ) form a primary key created , TIMESTAMP type, automatically set by SQL engine to current date/time.
NOTE: rows are automatically removed and updated (read about CASCADE ).
comments table commentid , integer, primary key owner , at most 20 chars, foreign key to users table postid , integer, foreign key to posts table text , at most 1024 chars created , TIMESTAMP type, automatically set by SQL engine to current date/time. NOTE: rows are automatically removed and updated (read about CASCADE ).
likes table owner , at most 20 chars, foreign key to users postid , integer, foreign key to posts created , TIMESTAMP type, automatically set by SQL engine to current date/time.
The tuple ( owner , postid ) form a primary key
NOTE: rows are automatically removed and updated (read about CASCADE ).
Data
Update sql/data.sql to add all initial data. You can find a complete dump of the initial data in insta485db-dump.txt. Your timestamps will be different. The passwords are all set to password .
Testing
Run the public autograder testcases on your database schema and data. You will need to complete the Flask Tutorial before running this test.
$ pytest -v tests/test_database_public.py
...
========================== 2 passed in 1.76 seconds ===========================
Make sure these tests pass before moving on. The other unit tests rely on a fully functionally bin/insta485db script.
Flask app
Complete the Flask Tutorial.
You should now have a directory containing an insta485 Python module.
$ tree insta485 insta485 ├── __init__.py ├── config.py
├── model.py
├── static
│ └── css
│ │ └── style.css
├── templates
│ └── index.html
└── views
├── __init__.py
└── index.py
The insta485run script starts a development server and you can browse to http://localhost:8000/ where you’ll see your “hello world” app.
$ ./bin/insta485run
+ test -e var/insta485.sqlite3
+ export FLASK_DEBUG=True
+ FLASK_DEBUG=True
+ export FLASK_APP=insta485
+ FLASK_APP=insta485
+ export INSTA485_SETTINGS=config.py
+ INSTA485_SETTINGS=config.py
+ flask run --host 0.0.0.0 --port 8000
* Serving Flask app "insta485" (lazy loading)
* Environment: production
WARNING: Do not use the development server in a production environment. Use a production WSGI server instead.
* Debug mode: on
* Running on http://0.0.0.0:8000/ (Press CTRL+C to quit)
Testing
Compliant HTML
Automatically generated HTML shall be W3C HTML5 compliant. To test dynamically generated pages, we’re going crawl the site using wget to download each page. Then, we’ll validate the pages using html5validator just like we did in project 1. Starting a second process in a script gets a little complicated, so we’ve provided a script in the starter files, bin/insta485test-html .
NOTE: bin/insta485test-html uses your bin/insta485run , so make sure that works correctly.
NOTE: bin/insta485test-html uses wget which sends HEAD requests, not GET requests.
NOTE: You will not pass this test until the /accounts/logout has been implemented.
$ ./bin/insta485test-html
...
========================================
./bin/insta485test-html
PASS
========================================
insta485test script
Similar to project 1, all Python code must be PEP8 compliant, comments shall be PEP257 compliant, and code shall pass a pylint static analysis.
Write another script called bin/insta485test that does this:
1. Stop on errors and prints commands
2. Run pycodestyle insta485
3. Run pydocstyle insta485
4. Run pylint --disable=cyclic-import insta485
5. Run all unit tests using pytest -v tests
6. Run the provided bin/insta485test-html script to validate your app’s html, as described above
Don’t forget to check for shell script pitfalls.
$ file bin/*
bin/insta485db: Bourne-Again shell script text executable, ASCII text bin/insta485run: Bourne-Again shell script text executable, ASCII text bin/insta485test: Bourne-Again shell script text executable, ASCII text bin/insta485test-html: Bourne-Again shell script text executable, ASCII text
Unit tests
Now that you have the framework of the project in place and your utility scripts written, we can run some of the unit tests. We have provided the public Autograder tests.
Run the public autograder testcases on your utility scripts.
$ pytest -v tests/test_scripts_public.py
...
========================== 6 passed in 1.16 seconds ===========================
Note: if you get deprecation warnings from third party libraries, check out the pytest tutorial deprecation warnings to suppress them.
Dynamic Server-side Insta485 specification
This project includes the same pages as project 1. The pages also include buttons to follow, unfollow, like, unlike and comment. We’ll also add pages for user account administration.
URLs
List of URLs from project 1, including screenshots with user awdeorio logged in.
/ screenshot
/u/<user_url_slug/ screenshot
/u/<user_url_slug/followers/ screenshot
/u/<user_url_slug/following/ screenshot
/p/<postid_slug/ screenshot
/explore/ screenshot List of new URLs:
/accounts/login/ screenshot (no user logged in)
/accounts/logout/ Immediately redirects to /accounts/login/ . No screenshot.
/accounts/create/ screenshot (no user logged in)
/accounts/delete/ screenshot
/accounts/edit/ screenshot
/accounts/password/ screenshot
Hint: When linking to pages or static files look into flask’s url_for() and send_from_directory() function.
All pages
Include a link to the main page / .
If not logged in, redirect to /accounts/login/ .
If logged in, include a link to /explore/ .
If logged in, include a link to /u/<user_url_slug/ where user_url_slug is the logged in user.
Access control
The server should accept POST requests from the logged in user only. To reject a request with a permissions error, use flask.abort(403) .
The following examples assume you have a (mostly) working Insta485 project with a freshly reset database.
Normal example
This example is just like using a web browser to fill in the HTML login form rendered on the login page and then navigating to / . Then, it deletes a post.
Use curl to sign in to Insta485. This command issues a POST request to /accounts/login/ with a username and password.
$ curl -X POST http://localhost:8000/accounts/login/ \
-F username=awdeorio \
-F password=password \
--cookie-jar cookies.txt
View the index / page by issuing a GET request.
$ curl -b cookies.txt http://localhost:8000/
... <index.html page content here
Delete an Insta485 post. The user awdeorio can successfully delete his own post by issuing a POST request.
$ curl -X POST http://localhost:8000/p/1/ \
-F postid=1 \
-F 'delete=delete this post' \ -b cookies.txt
Malicious example
Even though the “Delete post” button is hidden on posts that the logged in user doesn’t own, any user can use a tool like curl to send a POST request to try to delete an Insta485 post.
Try to delete a post created by jflinn using awdeorio ’s cookies. We get a 403 Forbidden error. This is a good thing!
$ curl -X POST http://localhost:8000/p/2/ \
-F postid=2 \
-F 'delete=delete this post' \ -b cookies.txt
...
<title403 Forbidden</title ...
Index /
screenshot
The index page should include all posts from the logged in user and all other users that the logged in user is following. The most recent post should be at the top. For each post:
Link to the post detail page /p/<postid_slug/ by clicking on the timestamp.
Link to the owner’s page /u/<user_url_slug/ by clicking on their username or profile picture.
Time since the post was created in human-readable format
Number of likes, using correct English
Comments, with owner’s username, oldest at the top
Link to the comment owner’s page /u/<user_url_slug/ by clicking on their username.
“like” or “unlike” button, pick the logical one Comment input and submission button
Form for “like” button
<!-- DO NOT CHANGE THIS (aside from where we say 'FIXME') -- <form action="" method="post" enctype="multipart/form-data"
<input type="hidden" name="postid" value="<FIXME postid HERE"/
<input type="submit" name="like" value="like"/ </form
Form for “unlike” button
<!-- DO NOT CHANGE THIS (aside from where we say 'FIXME') -- <form action="" method="post" enctype="multipart/form-data"
<input type="hidden" name="postid" value="<FIXME postid HERE"/
<input type="submit" name="unlike" value="unlike"/ </form
Form for “comment” button
<!-- DO NOT CHANGE THIS (aside from where we say 'FIXME') -- <form action="" method="post" enctype="multipart/form-data"
<input type="hidden" name="postid" value="<FIXME postid HERE"/
<input type="text" name="text"/
<input type="submit" name="comment" value="comment"/ </form
/u/<user_url_slug/ screenshot
Be sure to include username ( user_url_slug ) Relationship
“following” if the logged in user is following user_url_slug . Also include an “unfollow” button.
“not following” if the logged in user is not following user_url_slug . Also include a “follow” button.
Blank if logged in user == user_url_slug
Number of posts, with correct English
Number of followers, with correct English
Link to /u/<user_url_slug/followers/ Number following
Link to /u/<user_url_slug/following/
Name
A small image for each post
Clicking on the image links to /p/<postid_url_slug/ For a user’s own page, also include
Link to /accounts/edit/
Link to /accounts/logout/
File upload form for creating a new post
Note: creating a new post should not redirect the user
Form for follow button
<!-- DO NOT CHANGE THIS (aside from where we say 'FIXME') -- <form action="" method="post" enctype="multipart/form-data"
<input type="submit" name="follow" value="follow"/
<input type="hidden" name="username" value="<FIXME username"/ </form
Form for unfollow button
<!-- DO NOT CHANGE THIS (aside from where we say 'FIXME') -- <form action="" method="post" enctype="multipart/form-data"
<input type="submit" name="unfollow" value="unfollow"/
<input type="hidden" name="username" value="<FIXME username"/ </form
Form for file upload
<!-- DO NOT CHANGE THIS --
<form action="" method="post" enctype="multipart/form-data"
<input type="file" name="file"
<input type="submit" name="create_post" value="upload new post"/ </form
We use the sha256 hash algorithm to compute a hash of file contents. You can do this at the command line like this:
$ wget http://andrewdeorio.com/assets/headshot.jpg -O awdeorio.jpg
$ sha256sum awdeorio.jpg
e1a7c5c32973862ee15173b0259e3efdb6a391af awdeorio.jpg Here’s how to compute filenames in your Flask app:
import os import shutil import tempfile import flask
def sha256sum(filename):
"""Return sha256 hash of file content, similar to UNIX sha256sum.""" content = open(filename, 'rb').read() sha256_obj = hashlib.sha256(content) return sha256_obj.hexdigest()
# Save POST request's file object to a temp file dummy, temp_filename = tempfile.mkstemp() file = flask.request.files["file"] file.save(temp_filename)
# Compute filename
hash_txt = sha256sum(temp_filename) dummy, suffix = os.path.splitext(file.filename) hash_filename_basename = hash_txt + suffix hash_filename = os.path.join( insta485.app.config["UPLOAD_FOLDER"], hash_filename_basename
)
# Move temp file to permanent location shutil.move(temp_filename, hash_filename)
/u/<user_url_slug/followers/ screenshot
List the users that are following user_url_slug . For each, include:
Icon
Username, with link to /u/<username/
Relationship to logged in user
“following” if logged in user is following username. Also, an “unfollow” button. See above for HTML form.
“not following” if logged in user is not following username. Also, a “follow” button. See above for HTML form.
Blank if logged in user == username
/u/<user_url_slug/following/ screenshot
List the users that user_url_slug is following. For each, include:
Icon
Username, with link to /u/<username/ Relationship to logged in user
“following” if logged in user is following username. Also, an “unfollow” button. See above for HTML form.
“not following” if logged in user is not following username. Also, a “follow” button. See above for HTML form.
Blank if logged in user == username
/p/<postid_slug/
screenshot
This page shows one post. Include the same information for this one post as is shown on the main page / .
Include a “delete” button next to each comment owned by the logged in user.
<!-- DO NOT CHANGE THIS (aside from where we say 'FIXME') -- <form action="" method="post" enctype="multipart/form-data"
<input type="hidden" name="commentid" value="<FIXME commentid"/
<input type="submit" name="uncomment" value="delete"/ </form
Include a “delete this post” button if the post is owned by the logged in user. Delete the file from the filesystem. Delete everything in the database related to this post. Redirect to /u/<user_url_slug/ for the user whose post was deleted.
<!-- DO NOT CHANGE THIS (aside from where we say 'FIXME') -- <form action="" method="post" enctype="multipart/form-data"
<input type="hidden" name="postid" value="<FIXME postid"/
<input type="submit" name="delete" value="delete this post"/ </form
/explore/
screenshot
This page lists all users not that the logged in user is not following and includes:
Icon
Username with link to /u/<user_url_slug/
“follow” button
See above for HTML form
/accounts/login/ screenshot
If logged in, redirect to / .
Otherwise, include username and password inputs, and a login button. If someone tries to log in to a user that does not exist or provides incorrect credentials, abort(403) .
Also include a link to /accounts/create/ in the page.
Use this HTML form code. Feel free to style it and include placeholder s.
<!-- DO NOT CHANGE THIS (aside from styling) --
<form action="" method="post" enctype="multipart/form-data"
<input type="text" name="username"/
<input type="password" name="password"/
<input type="submit" value="login"/
</form
Warning: only store minimal information in a session cookie!
/accounts/logout/
Log out user on the server side. Immediately redirect to /accounts/login/ , which is why this is no screenshot.
/accounts/create/
screenshot
If a user is already logged in, redirect to /accounts/edit/ .
If a user tries to create an account with an existing username in the database, abort(409) - this is an HTTP code indicating a Conflict Error.
If a user tries to create an account with an empty string as the password, abort(400) - this is an HTTP code indicating a Bad Request.
If the account is created successfully, log the user in, and redirect to the main page / .
Also include a link to /accounts/login/ in the page.
HTML form. Style as you like.
<!-- DO NOT CHANGE THIS (aside from styling) --
<form action="" method="post" enctype="multipart/form-data"
<input type="file" name="file"
<input type="text" name="fullname"/
<input type="text" name="username"/
<input type="text" name="email"/
<input type="password" name="password"/
<input type="submit" name="signup" value="sign up"/ </form
See above for file upload and naming procedure.
A password entry in the database contains the algorithm, salt and password hash separated by $ . Use the sha512 algorithm like this:
import uuid import hashlib algorithm = 'sha512' salt = uuid.uuid4().hex hash_obj = hashlib.new(algorithm) password_salted = salt + password
hash_obj.update(password_salted.encode('utf-8')) password_hash = hash_obj.hexdigest() password_db_string = "$".join([algorithm, salt, password_hash]) print(password_db_string)
We’ve shipped one of the autograder tests so you can verify that login and logout mechanics work correctly. It’s in starter_files/tests/test_login_logout_public.py . Save it to
tests/test_login_logout_public.py and run it like this:
$ pytest -v tests/test_login_logout_public.py == test session starts ==
...
== 4 passed in 0.91 seconds ==
/accounts/delete/
screenshot
Confirmation page includes username and this form:
<!-- DO NOT CHANGE THIS --
<form action="" method="post" enctype="multipart/form-data"
<input type="submit" name="delete" value="confirm delete account"/ </form
Delete all post files created by this user. Delete user icon file. Delete all related entries in all tables. Hint: database tables set up properly with primary/foreign key relationships and CASCADE ON DELETE will do this automatically.
Upon successful submission, redirect to /accounts/create/ .
/accounts/edit/
screenshot
Include user’s icon and username, which are not editable.
Include a form with photo upload, name and email. Name and email are automatically filled in with previous value. If no photo file is included, then the server will update only the user’s name and email.
If a photo file is included, then the server will update the user’s photo, name and email. Delete the old photo from the filesystem. See above for file upload and naming procedure.
Link to /accounts/password/ .
Link to /accounts/delete/ .
Use this form
<!-- DO NOT CHANGE THIS (aside from where we say 'FIXME') -- <form action="" method="post" enctype="multipart/form-data"
<input type="file" name="file"
<input type="text" name="fullname" value="<FIXME full name here"/
<input type="text" name="email" value="<FIXME email here"/
<input type="submit" name="update" value="submit"/ </form
Upon successful submission, re-render the current page.
/accounts/password/
screenshot
Include this form:
<!-- DO NOT CHANGE THIS --
<form action="" method="post" enctype="multipart/form-data"
<input type="password" name="password"/
<input type="password" name="new_password1"/
<input type="password" name="new_password2"/ <input type="submit" name="update_password" value="submit"/ </form
Link to /accounts/edit/ .
Check the user’s password and abort(403) if it fails.
Check if both new passwords match. abort(401) otherwise.
Update hashed password entry in database. (See above).
Upon successful submission, redirect to /accounts/edit/ .
Static File Permissions
A user with the direct link to an uploaded file, /uploads/<filename , should only be able to see that file if logged in. If an unauthenticated user attempts to access an uploaded file, abort(403) .
Testing
Run the unit tests. Everything except for the deploy test should pass.
$ pytest -v
1 file (1.6MB)
