Lecture 24 Slides - Using Real World Data

1 of 19

Wider world of programming

1

CS 110

2 of 19

Reminders and Announcements

Announcements

  • P1 Grades posted this afternoon (median: 43 / 40)
  • P2 Available - SIGN UP FOR TEAMS BY TUESDAY @ 11:59PM IF YOU WANT
  • Common EoQ questions thread on edSTEM!

This Week

  • Monday - Python in the Real World
  • Wednesday - P2 Setup (Pre-Recorded/MQ14) + Tutorial 7: P2 Workshop
  • Friday - Q3 (See FAQ for questions about Emergency Grade Erasure)
    • Last moment to fix any grade discrepancies prior btw Q2 and Q3 is start of Q3
    • Those with testing accommodations have receive an email from me!

Finals Week

  • P2 due - Suggested Deadline: 3/18 at 5pm; Final Deadline: 3/20 at 5pm

3 of 19

Dictionaries (Cheat Sheet) - 01_dictionary_review.py

3

Create a dictionary

my_dict = {}

Get element from a dictionary

my_dict.get(key)

Add/replace items in dictionary

my_dict[key] = value

Remove items from dictionary

del my_dict[key]

Iterate through a dictionary

for key in my_dict:� print(key, my_dict[key])

Check if key exists in dictionary

if my_dict.get(key) != None

4 of 19

Dictionaries: When are they useful?

  • Lookups and crosswalks�Examples: language translation; lookup state capitals
  • Representation�Useful for representing complex entities via heterogeneous dictionaries�Examples: Twitter data (status, user, comment, etc.)
  • Counting and grouping�Useful for tallying, counting, and assigning information to categories�Examples: how many violations per restaurant; how many representatives per state

4

5 of 19

But where do we get data from?

5

6 of 19

  • A module is a file containing Python definitions and statements intended for use in other Python programs
  • There are many Python modules that come with Python as part of the standard library (e.g. operator, random, tkinter, datetime):
  • You can also make your own!

6

Recall what a module is...

7 of 19

Third-Party Modules

One of the reasons that Python is so great is because the broader Python community has built many, many useful libraries for different kinds of domain-specific computation, including:

  • Web scraping
  • Web Servers (e.g. Django, Flask)
  • Scientific Python (e.g. SciPy)
  • Numerical Python (e.g. NumPy)
  • Image processing (e.g. Pillow)
  • Music Processing (e.g. psonic)
  • Computer Vision (e.g. OpenCV bindings)
  • Deep Learning (e.g. PyTorch)
  • And more!

But how do you get them!?!?

7

8 of 19

Stuff that's just for fun

8

9 of 19

The Command Line

  • The command line provides another way to read, create, and execute files on your operating system
  • Anything you can do by using your Finder / File Explorer can also be done through your command line
  • Programmers (and online tutorials and documentation) often assume that you have some familiarity with the command line
  • Although it takes some getting used to, understanding where your files are stored and how to execute them from the command line (or IDLE) is an important programming skill

​

9

10 of 19

Window & Mac Have Different Conventions

The Windows and Mac Operating Systems use different languages to interact with the file system.

  • Windows uses something called MS-DOS
  • Mac and Linux machines use the Unix Shell
  • One big difference is using forward slashes or backslashes to specify a file path:
    • Windows uses backslashes \
    • Mac uses forward slashes /

10

11 of 19

The Command Line

  • Windows: DOS — uses the command prompt
  • Linux / Mac: Shell — uses Terminal
  • See https://bain-cs110.github.io/resources/using-pip cheat sheet for guidance...

​

11

12 of 19

Demo

  • Open your Terminal (Mac) or command prompt (Windows)
  • What directory are you in?
  • What files are in your directory?
  • Navigate to your cs110 folder on the command line
    • Note: if you have any spaces in your folder names, you have to surround your file path with quotation marks
  • Navigate to your lectures folder
  • Navigate back into your cs110 folder

Bonus

  • Try making a new directory in your cs110 folder from the command line.

12

13 of 19

Accessing Python from the Command Line

  • Open Terminal / Command Prompt
  • Type: python3
    • If you are taken to the python interpreter (immediate mode), then your python is already configured.
    • If not, you probably have a Windows machine. Navigate to cheat sheet linked a few files ago and go to “If you have a Windows machine”

13

14 of 19

The pip command is a tool for installing and managing Python packages

  • Developers publish their packages to pip, for others to share and re-use
  • You can search all of the available packages here.
  • We do this instead of manually downloading zip files
  • Pip is a recursive acronym for “Pip Installs Packages”
  • To install a package, open your command prompt and type the following from any directory (doesn’t matter):��pip3 install your_package_name

​

14

PIP

15 of 19

Modules Demos

Pretty Text

Wikipedia

Web servers

Sound

​

….to P2

15

16 of 19

Web APIs

  • You can think of Web APIs as functions that you access over the Internet using the http/https protocol.
  • They allow you to create, read, update, and delete data
  • Many organizations make their data available over REST (REpresentational State Transfer) so that third party apps (like your Project 2) can interoperate with their data

16

Your Computer

Server

APIs

17 of 19

Access and Authentication

  • In order to access data from a REST API, you typically need to register with the provider and get an authorization token.
  • In the following examples, we'll show how you might create functions that do this from scratch, in Project 2, we'll give you a library of functions that already exist

​

17

18 of 19

Speaking of existing functions…

18

19 of 19

Examples of Web APIs

Some examples of companies and organizations that have REST APIs include

  • Yelp
  • Spotify
  • CTA
  • Divvy Bikes
  • YouTube
  • Flickr

19