Part 1 – Web Scraping
Write a Python program that uses BeautifulSoup to go to
https://news.google.com and prints out all of the headlines on the page. Then, write a function called
find_headline_by_keyword which lets you search through those headlines for keywords, and returns to you a list of all of the headlines that match all the keywords you provide.
Part 2 – Web Scraping + File IO
This Wikipedia page has a table with data on all of the US Presidential elections. Our goal is to use Beautiful Soup to scrape some of this data into a CSV file. The columns of the CSV should be: order, year, winner, winner electoral votes, runner-up, and runner-up electoral votes. Use commas as the delimiter. For instance, after the header row, the first row of data should look like this:
1st,1788–1789,George Washington,69,John Adams,34
Hint: use the
pdb debugger! Setting break points is a great way to experiment with your code to make sure that you’re selecting the right elements and correctly targeting the text that you’re interested in.)
Part 3 – Server Side Requests
requests module and the OMDB API, build an application that prompts the user for two pieces of information, the name of an actor/actress and a movie. Your program should tell the user if that actor or actress was in that movie (this will only work for leading actors and actresses). As a bonus, add functionality to tell users who the director and writer of a movie were.
For solutions to these exercises, click here.