In [1]:
# There are dozens -- DOZENS -- of python-based Twitter libraries.
# It can be a challenge to find a good one that does everything you
# need.  Be sure to document your use of the library(ies) you choose:
# Libraries are most typically abandoned when data-owners release
# sweeping new versions of their API.  If you document the code
# well, it won't be hard to swap in a new library for the old.

# We'll use the tweepy library, which -- with the exception of
# its awful name -- is great in every way.

import tweepy
In [2]:
# Our friend json is in town.

import json

Papers, Please

In [3]:
#Fill in your access token and rename this file credentials.ini
#Get credentials here: https://apps.twitter.com
consumer_key='XXX'
consumer_key_secret='XXX'
access_token='XXX-XXX'
access_token_secret='XXX'

Tweepy helps us deal with OAuth2

In [4]:
myAuthorization = tweepy.OAuthHandler(consumer_key, consumer_key_secret)
In [5]:
myAuthorization.set_access_token(access_token, access_token_secret)

Now we create a researcherPackage

First: Give it credentials

In [6]:
researcherPackage = tweepy.API(myAuthorization)
In [7]:
tweet_list = []

Fly, little tweepy researcherPackage! Fly!

This .home_timeline() method flies out and grabs my account's twitter timeline.

In [8]:
raw_tweets = researcherPackage.home_timeline()

Examine the data returned

In [9]:
print(len(raw_tweets))
20
In [10]:
print(type(raw_tweets))
<class 'tweepy.models.ResultSet'>
In [11]:
# What does that raw .ResultSet look like?
# Let's look at ONLY the first (the zero-th) entry:

print(raw_tweets[0])
Status(_api=<tweepy.api.API object at 0x10c02eb38>, _json={'created_at': 'Mon Apr 09 01:24:36 +0000 2018', 'id': 983153673848131584, 'id_str': '983153673848131584', 'text': 'Why should anyone care whether blockchain or permissioned ledgers are a novel technology? The internet was a boring… https://t.co/Wpq7gN9JqT', 'truncated': True, 'entities': {'hashtags': [], 'symbols': [], 'user_mentions': [], 'urls': [{'url': 'https://t.co/Wpq7gN9JqT', 'expanded_url': 'https://twitter.com/i/web/status/983153673848131584', 'display_url': 'twitter.com/i/web/status/9…', 'indices': [117, 140]}]}, 'source': '<a href="https://about.twitter.com/products/tweetdeck" rel="nofollow">TweetDeck</a>', 'in_reply_to_status_id': None, 'in_reply_to_status_id_str': None, 'in_reply_to_user_id': None, 'in_reply_to_user_id_str': None, 'in_reply_to_screen_name': None, 'user': {'id': 817970, 'id_str': '817970', 'name': 'Kevin Werbach', 'screen_name': 'kwerb', 'location': 'Philly, USA', 'description': 'Wharton prof, tech policy maven, game thinker, MOOC instructor, blockchain researcher, pescatarian cook, feminist. Co-author of For the Win.', 'url': 'http://t.co/WiMmxYRPCl', 'entities': {'url': {'urls': [{'url': 'http://t.co/WiMmxYRPCl', 'expanded_url': 'http://werbach.com', 'display_url': 'werbach.com', 'indices': [0, 22]}]}, 'description': {'urls': []}}, 'protected': False, 'followers_count': 26194, 'friends_count': 1484, 'listed_count': 1442, 'created_at': 'Wed Mar 07 13:19:22 +0000 2007', 'favourites_count': 2152, 'utc_offset': -14400, 'time_zone': 'Eastern Time (US & Canada)', 'geo_enabled': True, 'verified': False, 'statuses_count': 13370, 'lang': 'en', 'contributors_enabled': False, 'is_translator': False, 'is_translation_enabled': False, 'profile_background_color': '250080', 'profile_background_image_url': 'http://pbs.twimg.com/profile_background_images/687246204/8a066bf59e4ff65ac0dfe00a13865f89.jpeg', 'profile_background_image_url_https': 'https://pbs.twimg.com/profile_background_images/687246204/8a066bf59e4ff65ac0dfe00a13865f89.jpeg', 'profile_background_tile': True, 'profile_image_url': 'http://pbs.twimg.com/profile_images/982255268976254977/ibn0izEn_normal.jpg', 'profile_image_url_https': 'https://pbs.twimg.com/profile_images/982255268976254977/ibn0izEn_normal.jpg', 'profile_banner_url': 'https://pbs.twimg.com/profile_banners/817970/1518149861', 'profile_link_color': '0000FF', 'profile_sidebar_border_color': 'FFFFFF', 'profile_sidebar_fill_color': 'E0FF92', 'profile_text_color': '000000', 'profile_use_background_image': True, 'has_extended_profile': True, 'default_profile': False, 'default_profile_image': False, 'following': True, 'follow_request_sent': False, 'notifications': False, 'translator_type': 'none'}, 'geo': None, 'coordinates': None, 'place': None, 'contributors': None, 'is_quote_status': False, 'retweet_count': 0, 'favorite_count': 0, 'favorited': False, 'retweeted': False, 'lang': 'en'}, created_at=datetime.datetime(2018, 4, 9, 1, 24, 36), id=983153673848131584, id_str='983153673848131584', text='Why should anyone care whether blockchain or permissioned ledgers are a novel technology? The internet was a boring… https://t.co/Wpq7gN9JqT', truncated=True, entities={'hashtags': [], 'symbols': [], 'user_mentions': [], 'urls': [{'url': 'https://t.co/Wpq7gN9JqT', 'expanded_url': 'https://twitter.com/i/web/status/983153673848131584', 'display_url': 'twitter.com/i/web/status/9…', 'indices': [117, 140]}]}, source='TweetDeck', source_url='https://about.twitter.com/products/tweetdeck', in_reply_to_status_id=None, in_reply_to_status_id_str=None, in_reply_to_user_id=None, in_reply_to_user_id_str=None, in_reply_to_screen_name=None, author=User(_api=<tweepy.api.API object at 0x10c02eb38>, _json={'id': 817970, 'id_str': '817970', 'name': 'Kevin Werbach', 'screen_name': 'kwerb', 'location': 'Philly, USA', 'description': 'Wharton prof, tech policy maven, game thinker, MOOC instructor, blockchain researcher, pescatarian cook, feminist. Co-author of For the Win.', 'url': 'http://t.co/WiMmxYRPCl', 'entities': {'url': {'urls': [{'url': 'http://t.co/WiMmxYRPCl', 'expanded_url': 'http://werbach.com', 'display_url': 'werbach.com', 'indices': [0, 22]}]}, 'description': {'urls': []}}, 'protected': False, 'followers_count': 26194, 'friends_count': 1484, 'listed_count': 1442, 'created_at': 'Wed Mar 07 13:19:22 +0000 2007', 'favourites_count': 2152, 'utc_offset': -14400, 'time_zone': 'Eastern Time (US & Canada)', 'geo_enabled': True, 'verified': False, 'statuses_count': 13370, 'lang': 'en', 'contributors_enabled': False, 'is_translator': False, 'is_translation_enabled': False, 'profile_background_color': '250080', 'profile_background_image_url': 'http://pbs.twimg.com/profile_background_images/687246204/8a066bf59e4ff65ac0dfe00a13865f89.jpeg', 'profile_background_image_url_https': 'https://pbs.twimg.com/profile_background_images/687246204/8a066bf59e4ff65ac0dfe00a13865f89.jpeg', 'profile_background_tile': True, 'profile_image_url': 'http://pbs.twimg.com/profile_images/982255268976254977/ibn0izEn_normal.jpg', 'profile_image_url_https': 'https://pbs.twimg.com/profile_images/982255268976254977/ibn0izEn_normal.jpg', 'profile_banner_url': 'https://pbs.twimg.com/profile_banners/817970/1518149861', 'profile_link_color': '0000FF', 'profile_sidebar_border_color': 'FFFFFF', 'profile_sidebar_fill_color': 'E0FF92', 'profile_text_color': '000000', 'profile_use_background_image': True, 'has_extended_profile': True, 'default_profile': False, 'default_profile_image': False, 'following': True, 'follow_request_sent': False, 'notifications': False, 'translator_type': 'none'}, id=817970, id_str='817970', name='Kevin Werbach', screen_name='kwerb', location='Philly, USA', description='Wharton prof, tech policy maven, game thinker, MOOC instructor, blockchain researcher, pescatarian cook, feminist. Co-author of For the Win.', url='http://t.co/WiMmxYRPCl', entities={'url': {'urls': [{'url': 'http://t.co/WiMmxYRPCl', 'expanded_url': 'http://werbach.com', 'display_url': 'werbach.com', 'indices': [0, 22]}]}, 'description': {'urls': []}}, protected=False, followers_count=26194, friends_count=1484, listed_count=1442, created_at=datetime.datetime(2007, 3, 7, 13, 19, 22), favourites_count=2152, utc_offset=-14400, time_zone='Eastern Time (US & Canada)', geo_enabled=True, verified=False, statuses_count=13370, lang='en', contributors_enabled=False, is_translator=False, is_translation_enabled=False, profile_background_color='250080', profile_background_image_url='http://pbs.twimg.com/profile_background_images/687246204/8a066bf59e4ff65ac0dfe00a13865f89.jpeg', profile_background_image_url_https='https://pbs.twimg.com/profile_background_images/687246204/8a066bf59e4ff65ac0dfe00a13865f89.jpeg', profile_background_tile=True, profile_image_url='http://pbs.twimg.com/profile_images/982255268976254977/ibn0izEn_normal.jpg', profile_image_url_https='https://pbs.twimg.com/profile_images/982255268976254977/ibn0izEn_normal.jpg', profile_banner_url='https://pbs.twimg.com/profile_banners/817970/1518149861', profile_link_color='0000FF', profile_sidebar_border_color='FFFFFF', profile_sidebar_fill_color='E0FF92', profile_text_color='000000', profile_use_background_image=True, has_extended_profile=True, default_profile=False, default_profile_image=False, following=True, follow_request_sent=False, notifications=False, translator_type='none'), user=User(_api=<tweepy.api.API object at 0x10c02eb38>, _json={'id': 817970, 'id_str': '817970', 'name': 'Kevin Werbach', 'screen_name': 'kwerb', 'location': 'Philly, USA', 'description': 'Wharton prof, tech policy maven, game thinker, MOOC instructor, blockchain researcher, pescatarian cook, feminist. Co-author of For the Win.', 'url': 'http://t.co/WiMmxYRPCl', 'entities': {'url': {'urls': [{'url': 'http://t.co/WiMmxYRPCl', 'expanded_url': 'http://werbach.com', 'display_url': 'werbach.com', 'indices': [0, 22]}]}, 'description': {'urls': []}}, 'protected': False, 'followers_count': 26194, 'friends_count': 1484, 'listed_count': 1442, 'created_at': 'Wed Mar 07 13:19:22 +0000 2007', 'favourites_count': 2152, 'utc_offset': -14400, 'time_zone': 'Eastern Time (US & Canada)', 'geo_enabled': True, 'verified': False, 'statuses_count': 13370, 'lang': 'en', 'contributors_enabled': False, 'is_translator': False, 'is_translation_enabled': False, 'profile_background_color': '250080', 'profile_background_image_url': 'http://pbs.twimg.com/profile_background_images/687246204/8a066bf59e4ff65ac0dfe00a13865f89.jpeg', 'profile_background_image_url_https': 'https://pbs.twimg.com/profile_background_images/687246204/8a066bf59e4ff65ac0dfe00a13865f89.jpeg', 'profile_background_tile': True, 'profile_image_url': 'http://pbs.twimg.com/profile_images/982255268976254977/ibn0izEn_normal.jpg', 'profile_image_url_https': 'https://pbs.twimg.com/profile_images/982255268976254977/ibn0izEn_normal.jpg', 'profile_banner_url': 'https://pbs.twimg.com/profile_banners/817970/1518149861', 'profile_link_color': '0000FF', 'profile_sidebar_border_color': 'FFFFFF', 'profile_sidebar_fill_color': 'E0FF92', 'profile_text_color': '000000', 'profile_use_background_image': True, 'has_extended_profile': True, 'default_profile': False, 'default_profile_image': False, 'following': True, 'follow_request_sent': False, 'notifications': False, 'translator_type': 'none'}, id=817970, id_str='817970', name='Kevin Werbach', screen_name='kwerb', location='Philly, USA', description='Wharton prof, tech policy maven, game thinker, MOOC instructor, blockchain researcher, pescatarian cook, feminist. Co-author of For the Win.', url='http://t.co/WiMmxYRPCl', entities={'url': {'urls': [{'url': 'http://t.co/WiMmxYRPCl', 'expanded_url': 'http://werbach.com', 'display_url': 'werbach.com', 'indices': [0, 22]}]}, 'description': {'urls': []}}, protected=False, followers_count=26194, friends_count=1484, listed_count=1442, created_at=datetime.datetime(2007, 3, 7, 13, 19, 22), favourites_count=2152, utc_offset=-14400, time_zone='Eastern Time (US & Canada)', geo_enabled=True, verified=False, statuses_count=13370, lang='en', contributors_enabled=False, is_translator=False, is_translation_enabled=False, profile_background_color='250080', profile_background_image_url='http://pbs.twimg.com/profile_background_images/687246204/8a066bf59e4ff65ac0dfe00a13865f89.jpeg', profile_background_image_url_https='https://pbs.twimg.com/profile_background_images/687246204/8a066bf59e4ff65ac0dfe00a13865f89.jpeg', profile_background_tile=True, profile_image_url='http://pbs.twimg.com/profile_images/982255268976254977/ibn0izEn_normal.jpg', profile_image_url_https='https://pbs.twimg.com/profile_images/982255268976254977/ibn0izEn_normal.jpg', profile_banner_url='https://pbs.twimg.com/profile_banners/817970/1518149861', profile_link_color='0000FF', profile_sidebar_border_color='FFFFFF', profile_sidebar_fill_color='E0FF92', profile_text_color='000000', profile_use_background_image=True, has_extended_profile=True, default_profile=False, default_profile_image=False, following=True, follow_request_sent=False, notifications=False, translator_type='none'), geo=None, coordinates=None, place=None, contributors=None, is_quote_status=False, retweet_count=0, favorite_count=0, favorited=False, retweeted=False, lang='en')
In [12]:
# Wow.  So, this prompts a question:
# If a tweet is typically 140 characters, how big
# is the (tweepy) raw data behind that tweet?

# I'll break this into a few steps:
In [13]:
# Get 1 raw tweet only
stepOne = raw_tweets[0]
In [14]:
# Did that change the type of data?  Nope.
print(type(stepOne))
<class 'tweepy.models.Status'>
In [15]:
# So we need to deal with Strings, not models.
# Change tweepy.models.Status object to a simple String
stepTwo = str(stepOne)
In [16]:
# Now that it is just a String of text, 
# get the length of that simple String
stepThree = len(stepTwo)
In [17]:
# Let's see what we got
print(stepThree)

# What does that mean in terms of a ratio of
# infrastructure to visible data?

output = str(int(stepThree/140))
print(output + " : 1 ratio, infrastructure : visible")
11250
80 : 1 ratio, infrastructure : visible
In [18]:
# Again, wow.
# One more thought before we switch up:
# Don't forget that I'm always dragging things out
# In a Python environment, I could get the same
# result by typing this:

len(str(raw_tweets[0]))
Out[18]:
11250

More Than This

Tweepy's format is not quite JSON.

Tweepy does take Twitter's JSON-encoded tweets and add some more information.

This means that they aren't as easy to use as they could be. But Tweepy has a tool (a method) built into it that will (mostly) fix that for us. Remember the data science rule of thumb: As much as 90% of your time will be spent massaging your dataset.

In [19]:
JSONTweet = stepOne._json
print(type(JSONTweet))
<class 'dict'>

Its a dict! A dict! Its just KVP [Key Value Pairs], which are always stored in a predictable List! Like this:

[(K,V), (K,V), (K,V)]

In [20]:
for key in JSONTweet:
    value = JSONTweet[key]
    print(key,'-->',value)
created_at --> Mon Apr 09 01:24:36 +0000 2018
id --> 983153673848131584
id_str --> 983153673848131584
text --> Why should anyone care whether blockchain or permissioned ledgers are a novel technology? The internet was a boring… https://t.co/Wpq7gN9JqT
truncated --> True
entities --> {'hashtags': [], 'symbols': [], 'user_mentions': [], 'urls': [{'url': 'https://t.co/Wpq7gN9JqT', 'expanded_url': 'https://twitter.com/i/web/status/983153673848131584', 'display_url': 'twitter.com/i/web/status/9…', 'indices': [117, 140]}]}
source --> <a href="https://about.twitter.com/products/tweetdeck" rel="nofollow">TweetDeck</a>
in_reply_to_status_id --> None
in_reply_to_status_id_str --> None
in_reply_to_user_id --> None
in_reply_to_user_id_str --> None
in_reply_to_screen_name --> None
user --> {'id': 817970, 'id_str': '817970', 'name': 'Kevin Werbach', 'screen_name': 'kwerb', 'location': 'Philly, USA', 'description': 'Wharton prof, tech policy maven, game thinker, MOOC instructor, blockchain researcher, pescatarian cook, feminist. Co-author of For the Win.', 'url': 'http://t.co/WiMmxYRPCl', 'entities': {'url': {'urls': [{'url': 'http://t.co/WiMmxYRPCl', 'expanded_url': 'http://werbach.com', 'display_url': 'werbach.com', 'indices': [0, 22]}]}, 'description': {'urls': []}}, 'protected': False, 'followers_count': 26194, 'friends_count': 1484, 'listed_count': 1442, 'created_at': 'Wed Mar 07 13:19:22 +0000 2007', 'favourites_count': 2152, 'utc_offset': -14400, 'time_zone': 'Eastern Time (US & Canada)', 'geo_enabled': True, 'verified': False, 'statuses_count': 13370, 'lang': 'en', 'contributors_enabled': False, 'is_translator': False, 'is_translation_enabled': False, 'profile_background_color': '250080', 'profile_background_image_url': 'http://pbs.twimg.com/profile_background_images/687246204/8a066bf59e4ff65ac0dfe00a13865f89.jpeg', 'profile_background_image_url_https': 'https://pbs.twimg.com/profile_background_images/687246204/8a066bf59e4ff65ac0dfe00a13865f89.jpeg', 'profile_background_tile': True, 'profile_image_url': 'http://pbs.twimg.com/profile_images/982255268976254977/ibn0izEn_normal.jpg', 'profile_image_url_https': 'https://pbs.twimg.com/profile_images/982255268976254977/ibn0izEn_normal.jpg', 'profile_banner_url': 'https://pbs.twimg.com/profile_banners/817970/1518149861', 'profile_link_color': '0000FF', 'profile_sidebar_border_color': 'FFFFFF', 'profile_sidebar_fill_color': 'E0FF92', 'profile_text_color': '000000', 'profile_use_background_image': True, 'has_extended_profile': True, 'default_profile': False, 'default_profile_image': False, 'following': True, 'follow_request_sent': False, 'notifications': False, 'translator_type': 'none'}
geo --> None
coordinates --> None
place --> None
contributors --> None
is_quote_status --> False
retweet_count --> 0
favorite_count --> 0
favorited --> False
retweeted --> False
lang --> en

Worth reminding you that since KVPs are so important, Python 3 introduced something I call Easy Key Value Pairs for Python Three, or EZKVP4PIII.

To wit, in lieu of this:

for key in JSONTweet: value = JSONTweet[key] print(key,'-->',value)

We can use the .items() method, which returns the paired values of a dictionary, on after another.

Explainer: Above, JSONTweet is a dictionary. We ask it to iterate through all the keys in that dictionary. Every time we get a key, we "look up" the value of that key. It is a bit redundant.

Explainer: Below, JSONTweet.items() "breaks apart" the dictionary into a list of all its Key Value pairs, which it delivers to us, one after another. Because both the key and the value are delivered to us inside a "tuple" (k,v), we don't have to ask Python to look up every key.

In [21]:
for key, value in JSONTweet.items():
    print(key,'-->',value)
created_at --> Mon Apr 09 01:24:36 +0000 2018
id --> 983153673848131584
id_str --> 983153673848131584
text --> Why should anyone care whether blockchain or permissioned ledgers are a novel technology? The internet was a boring… https://t.co/Wpq7gN9JqT
truncated --> True
entities --> {'hashtags': [], 'symbols': [], 'user_mentions': [], 'urls': [{'url': 'https://t.co/Wpq7gN9JqT', 'expanded_url': 'https://twitter.com/i/web/status/983153673848131584', 'display_url': 'twitter.com/i/web/status/9…', 'indices': [117, 140]}]}
source --> <a href="https://about.twitter.com/products/tweetdeck" rel="nofollow">TweetDeck</a>
in_reply_to_status_id --> None
in_reply_to_status_id_str --> None
in_reply_to_user_id --> None
in_reply_to_user_id_str --> None
in_reply_to_screen_name --> None
user --> {'id': 817970, 'id_str': '817970', 'name': 'Kevin Werbach', 'screen_name': 'kwerb', 'location': 'Philly, USA', 'description': 'Wharton prof, tech policy maven, game thinker, MOOC instructor, blockchain researcher, pescatarian cook, feminist. Co-author of For the Win.', 'url': 'http://t.co/WiMmxYRPCl', 'entities': {'url': {'urls': [{'url': 'http://t.co/WiMmxYRPCl', 'expanded_url': 'http://werbach.com', 'display_url': 'werbach.com', 'indices': [0, 22]}]}, 'description': {'urls': []}}, 'protected': False, 'followers_count': 26194, 'friends_count': 1484, 'listed_count': 1442, 'created_at': 'Wed Mar 07 13:19:22 +0000 2007', 'favourites_count': 2152, 'utc_offset': -14400, 'time_zone': 'Eastern Time (US & Canada)', 'geo_enabled': True, 'verified': False, 'statuses_count': 13370, 'lang': 'en', 'contributors_enabled': False, 'is_translator': False, 'is_translation_enabled': False, 'profile_background_color': '250080', 'profile_background_image_url': 'http://pbs.twimg.com/profile_background_images/687246204/8a066bf59e4ff65ac0dfe00a13865f89.jpeg', 'profile_background_image_url_https': 'https://pbs.twimg.com/profile_background_images/687246204/8a066bf59e4ff65ac0dfe00a13865f89.jpeg', 'profile_background_tile': True, 'profile_image_url': 'http://pbs.twimg.com/profile_images/982255268976254977/ibn0izEn_normal.jpg', 'profile_image_url_https': 'https://pbs.twimg.com/profile_images/982255268976254977/ibn0izEn_normal.jpg', 'profile_banner_url': 'https://pbs.twimg.com/profile_banners/817970/1518149861', 'profile_link_color': '0000FF', 'profile_sidebar_border_color': 'FFFFFF', 'profile_sidebar_fill_color': 'E0FF92', 'profile_text_color': '000000', 'profile_use_background_image': True, 'has_extended_profile': True, 'default_profile': False, 'default_profile_image': False, 'following': True, 'follow_request_sent': False, 'notifications': False, 'translator_type': 'none'}
geo --> None
coordinates --> None
place --> None
contributors --> None
is_quote_status --> False
retweet_count --> 0
favorite_count --> 0
favorited --> False
retweeted --> False
lang --> en

Also worth reminding you: Mother's Day is May 13th this year.

To return to our JSONTweet: Once it is JSON-friendly, we can easily extract data from it (as in our KVP demo above):

In [22]:
print(JSONTweet['text'])
Why should anyone care whether blockchain or permissioned ledgers are a novel technology? The internet was a boring… https://t.co/Wpq7gN9JqT

What fun, amirite? Let's put all of that into a single FOR loop, putting each new tweet into a List called tweet_list.

In [23]:
tweet_list = []
for each_tweet in raw_tweets:
    
    # convert it to a JSON friendly format
    # using a tool (method) built into Tweepy
    # called ._json
    
    JSONTweet = each_tweet._json
    
    # Let's extract the value stored
    # in the 'text' key of JSONTweet
    
    one_tweet = JSONTweet['text']
    print(one_tweet)
    # Let's add it to my ongoing List
    #print(one_tweet)
    tweet_list.append(one_tweet)
Why should anyone care whether blockchain or permissioned ledgers are a novel technology? The internet was a boring… https://t.co/Wpq7gN9JqT
I got a kick out of seeing that was actually 10 years ago. Also sidenote, Stermy was one of the few esports players… https://t.co/uYmcGZ4aru
Checked my files and actually had a couple pics of the FIFA 2008 CGS stuff. It was Stermy v Bazza. Kat Hunter was t… https://t.co/XTnfQPWLyD
What's that color? This sensor on sale in our store 'grabs' colors from real life, then tells you what RGB, CMYK, o… https://t.co/T1lKLEww8e
Awesome! Congrats on finding the tower! I hope@you saw what was inside as well. https://t.co/hgflDf0erC
66 Young Black Elected Officials Release National Agenda to Combat Police Brutality https://t.co/AfLG2egqoj #enoughisenough
like the cosmopolitan neighbor in Terence, Yoki the Oakland warehouse cat says with elegance, "nothing which is hum… https://t.co/rejT7jiwiO
RT @TomZohar: Love spending a relaxing Sunday curled up with a good book as it sits next to me untouched while I scroll through twitter for…
RT @benepstein55: My book, The Only Constant is Change: Technology, Political Communication and Innovation Over Time, just came out this we…
RT @AxiosWorld: From @BarakRavid: Netanyahu tried and failed to convince Trump to rethink his decision to withdraw from Syria. https://t.co…
OMG #RHOAReunion https://t.co/ihJKnUtTIY
RT @marthasjones_: This thread on lynching and its memory, from @Sifill_LDF. https://t.co/JGwi18NVGb
RT @erin_bartram: This is the Wineburg-inspired centerpiece of the class. Reading 22 of these a semester will chasten you. https://t.co/CDT…
#RHOA https://t.co/sXEpQkkQ4i
RT @NPR: What happens after the worst natural disaster in Sierra Leone's history? The world moves on, and survivors face a harsh new realit…
Trump lashes out at Kirstjen Nielsen and blames her for the uptick in illegal crossings of the southern border. https://t.co/ArndxGZepy
LOOOOOL https://t.co/CEAblwnrNL
RT @poetastrologers: Saying hello to a Pisces https://t.co/tgVjXHbAQI
RT @poetastrologers: A Taurus saying I love you https://t.co/Cw5AMU9OJm
RT @jonathanvswan: NEW: Kirstjen Nielsen becomes Trump's immigration scapegoat... &amp; some new details: the week after she was nominated Trum…

The result? Pure gold, G.

PUREGOLD #FEMINISM #SQWADGOLS

In [24]:
print(tweet_list)
['Why should anyone care whether blockchain or permissioned ledgers are a novel technology? The internet was a boring… https://t.co/Wpq7gN9JqT', 'I got a kick out of seeing that was actually 10 years ago. Also sidenote, Stermy was one of the few esports players… https://t.co/uYmcGZ4aru', 'Checked my files and actually had a couple pics of the FIFA 2008 CGS stuff. It was Stermy v Bazza. Kat Hunter was t… https://t.co/XTnfQPWLyD', "What's that color? This sensor on sale in our store 'grabs' colors from real life, then tells you what RGB, CMYK, o… https://t.co/T1lKLEww8e", 'Awesome! Congrats on finding the tower! I hope@you saw what was inside as well. https://t.co/hgflDf0erC', '66 Young Black Elected Officials Release National Agenda to Combat Police Brutality https://t.co/AfLG2egqoj #enoughisenough', 'like the cosmopolitan neighbor in Terence, Yoki the Oakland warehouse cat says with elegance, "nothing which is hum… https://t.co/rejT7jiwiO', 'RT @TomZohar: Love spending a relaxing Sunday curled up with a good book as it sits next to me untouched while I scroll through twitter for…', 'RT @benepstein55: My book, The Only Constant is Change: Technology, Political Communication and Innovation Over Time, just came out this we…', 'RT @AxiosWorld: From @BarakRavid: Netanyahu tried and failed to convince Trump to rethink his decision to withdraw from Syria. https://t.co…', 'OMG #RHOAReunion https://t.co/ihJKnUtTIY', 'RT @marthasjones_: This thread on lynching and its memory, from @Sifill_LDF. https://t.co/JGwi18NVGb', 'RT @erin_bartram: This is the Wineburg-inspired centerpiece of the class. Reading 22 of these a semester will chasten you. https://t.co/CDT…', '#RHOA https://t.co/sXEpQkkQ4i', "RT @NPR: What happens after the worst natural disaster in Sierra Leone's history? The world moves on, and survivors face a harsh new realit…", 'Trump lashes out at Kirstjen Nielsen and blames her for the uptick in illegal crossings of the southern border. https://t.co/ArndxGZepy', 'LOOOOOL https://t.co/CEAblwnrNL', 'RT @poetastrologers: Saying hello to a Pisces https://t.co/tgVjXHbAQI', 'RT @poetastrologers: A Taurus saying I love you https://t.co/Cw5AMU9OJm', "RT @jonathanvswan: NEW: Kirstjen Nielsen becomes Trump's immigration scapegoat... &amp; some new details: the week after she was nominated Trum…"]

Again: We can take a single element from that List by calling for a specific element number. Recall that element numbers begin with 0.

In [25]:
print(tweet_list[8])
RT @benepstein55: My book, The Only Constant is Change: Technology, Political Communication and Innovation Over Time, just came out this we…

Cursors! Foiled Again!

One of the nicest things about Tweepy is also a little bit hard to get your head around at first, so I'm just going to mention it and leave it for you to consider later.

Tweepy makes use of a virtual cursor in order to sort through and display our gobs and gobs of data. Why? Because Twitter's API does out tweets in clumps called "Pages" -- e.g., 20 tweets per page. That makes sense on a web page, but for data collection it is a headache.

Even so, it isn't as simple as it ought to be. Here's an example of what it would look like to iterate through 5 tweets. (Note that this doesn't really start to take advantage of what Cursor really does, since I'm only using 5 tweets, but time is short. Note too that my explanation here is not much of an explanation, but -- again -- time's wingèd chariot, etc., etc.)

Let's take my tweepy researcherPackage who is already full of messages from her last outing (see above for a quick reminder).

I'll call that bundle of researcherPackage messages "message_block_one." Recall that our researcherPackage is a Tweepy object -- created when we first gave our authorization information to Tweepy.

Now I'll create what I call a Tweepy "collection" using Tweepy's Cursor method. But because the .Cursor() works in volume -- fetching lots of tweets -- I need to include a quantity to keep from hitting my limit. So I'll introduce the quantity of 3 via .items() -- which is EXACTLY the same as the one used in the iterating structure I covered above ("EZ KVP 4 Python III").

As I say -- it is a lot to take in. Look at the example below. Play with it if you like, or move on with me to the next part.

In [26]:
tweepyCollection = tweepy.Cursor(researcherPackage.home_timeline).items(5)
In [27]:
# Trickiest here?  When I say "each_tweet.text" I'm
# not using .text as a built-in function or method to
# convert the value to a text value (as often happens).
# Instead, I'm asking for the VALUE associated with
# a specific KEY, "text".  I could ask for other keys, too,
# like "verified", "retweeted", "followers", etc.

for each_tweet in tweepyCollection:    
    a_tweet = each_tweet.text
    print(a_tweet)
    print(" ")
Why should anyone care whether blockchain or permissioned ledgers are a novel technology? The internet was a boring… https://t.co/Wpq7gN9JqT
 
I got a kick out of seeing that was actually 10 years ago. Also sidenote, Stermy was one of the few esports players… https://t.co/uYmcGZ4aru
 
Checked my files and actually had a couple pics of the FIFA 2008 CGS stuff. It was Stermy v Bazza. Kat Hunter was t… https://t.co/XTnfQPWLyD
 
What's that color? This sensor on sale in our store 'grabs' colors from real life, then tells you what RGB, CMYK, o… https://t.co/T1lKLEww8e
 
Awesome! Congrats on finding the tower! I hope@you saw what was inside as well. https://t.co/hgflDf0erC
 
In [28]:
# Bear in mind Tweepy isn't the only library;
# And understand, too, that we don't necessarily NEED a library
# because Twitter already provides us an API.
# But researchers often feel that a specific
# API built for the peculiar qualities of a specific language
# can be useful.

# In Tweepy's case, for example, the developers
# decided that the search function inside Twitter was
# already good enough:  They didn't want to duplicate it.
# So we'll use it next, in tandem with our tweepy library.
In [29]:
# REMINDER:
# We're scholars, researchers, looking to build datasets
# But these tools are mostly for developers building applications

# So in Tweepy, for example, there are features we're unlikely
# to use in our capacities as scholars.  Like create a tweet,
# or destroy friendship, etc.
In [30]:
# Similarly, when Twitter built their API, they built it 
# with lots of conditionals and options that you can
# set in order to make your life easier.  IF you want to
# do a search of all the tweets posted today, for example,
# there's no reason you'd have to identify who you were
# as a USER, really:  Twitter just wants to know that
# you are legit, and then it lets you go forward.
# If, conversely, we wanted to search MY timeline for
# subversive anti-canadian provocations, for example, 
# Twitter might want to know more about you before you
# run that query.  Are you a friend of mine, for example?
# Did someone with my USERID give your account permission
# to access everything I've said?  If yes, fine.  But if
# no, then you're out of luck.  
In [31]:
# All of which is to say that your ability to access certain
# data may depend on what permissions you've been granted.

# Why does this matter to you?  Because if we identify ourselves
# to Twitter as "just a researcher doing her research", Twitter
# will allow us to make a certain number of inquiries.  But
# the more specific our inquiries become, the more work we
# are asking Twitter to do, so the fewer questions it allows us.

# User-authenticated searches top out at 180/15 mins
# But App-authenticated searches at 450/15 mins

# 450 searches returns (potentially) 45,000 tweets
# (since each query can return up to 100 tweets).
In [32]:
# NONE OF THIS MATTERED A CENTURY AGO.  THIS IS NEW.
# THIS IS SCALE.  THINGS HAPPEN ON A COMPLETELY DIFFERENT SCALE
# HERE, AND IT IS OUR RESPONSIBILITY TO UNDERSTAND (AS BEST WE CAN)
# THE MYRIAD WAYS IN WHICH THAT MAKES A DIFFERENCE.
In [33]:
# So use AppAuth handler instead of OAuthHandler for more Qs
authorization = tweepy.AppAuthHandler(consumer_key, consumer_key_secret)
In [34]:
# Scale matters.  If we exceed our quota, we want to know.
# Otherwise, if we exceed quota and keep hammering away
# at Twitter's API, Twitter may grow unhappy and revoke
# our credentials, etc.  We HAVE to pay attention to
# the data that WE CANNOT SEE.
In [35]:
researcherPackage.rate_limit_status()['resources']['search']
Out[35]:
{'/search/tweets': {'limit': 180, 'remaining': 180, 'reset': 1523238080}}
In [36]:
# tell tweepy how to behave by passing extra arguments

# here, I'm going to use a new "researcherPackage", but I'll
# call it just 'api' -- which is what you're likely to see
# other programmers call it.  NOTE (sigh) that api and API
# are very different things...

api = tweepy.API(authorization,wait_on_rate_limit=True,wait_on_rate_limit_notify=True)
In [37]:
api.rate_limit_status()['resources']['search']
Out[37]:
{'/search/tweets': {'limit': 450, 'remaining': 450, 'reset': 1523238080}}
In [38]:
# That query tells us where we're at.  The reset value
# refers to a time 15 minutes from now.
In [39]:
# Seriously, though:  Don't carry with you the hope that you'll crack this
# code and come to understand all of this like you understand, say, organ
# music from the German Baroque or the 2010 Pittsburgh Steelers offense or
# the role of adenosine triphosphate in intracellular energy transfer.  That
# kind of knowledge (total, embodied, comprehensive) just isn't useful to you here.
# And it certainly isn't practical.  Instead, learn how to identify
# code that is self-contained and well-documented.  If what you have
# built works, that is enough:  Don't be distracted by others' obsessions
# with efficiency or best practices.
In [40]:
# When anyone in America tweets #MAGA, we
# will look to find out if we can locate them
# on a map.  We need to look at lots of tweets,
# because most of them don't contain easily-
# accessed location information.

# For information of geo locations, you'll
# need to look at the Twitter documentation
# or 3rd-party documentation of the 
# Twitter API.  I mentioned in class that this
# data was per Yahoo's standards -- that is wrong.
# Yahoo uses something called woeid -- where on
# earth id -- which is totally different.

# I will note with disdain that inventing a numeric
# code with which to identify countries, cities, and
# landmarks is really only useful WHEN YOU MAKE THOSE
# CODES EASILY AVAILABLE TO THE PEOPLE THAT LIVE
# IN THOSE COUNTRIES and CITIES.  Otherwise, it is
# another identifier that is accessible only by
# insiders and elites.  THAT IS NOT SATISFACTORY.

searchPlace = 'place:96683cc9126741d1' # United States
searchQuery = 'college'
mySearch = searchPlace + " " + searchQuery
In [41]:
# borrowed from previous week's gmaps tutorials.

import gmaps
myKey = "XXX"
gmaps.configure(api_key=myKey)
myMap = gmaps.figure()
In [42]:
# Recall that to plot points on a map, 
# gmaps wants a List comprising LONG, LAT pairs:
# [(long,lat),(long,lat),(long,lat)]
In [43]:
tweet_origins =[]
tweet_text=[]
tweet_total=[]

# don't forget:  500 items (below) is the number of items
# returned, not necessarily the number of tweets
# that will end up on our map.
In [44]:
for each_Tweet in tweepy.Cursor(api.search,q=mySearch).items(500):
    this_Tweet = each_Tweet._json
    # the coordinates are often blank.  Is that the case?
    # If it is, it is not worth mapping 0,0.
    
    if this_Tweet['coordinates'] is not None:
        # OK:  So if we got to this part, it is
        # because the coordinates list isn't empty.
        # But the trick is this:  There is another
        # list INSIDE the coordinates list.  And 
        # its name is... coordinates.  Sigh.  So
        # we have to ask for the coordinates list
        # inside the coordinates list.  That should
        # give us a list of two numbers.
        location_pair = this_Tweet['coordinates']['coordinates']
        
        # I take the list apart, only to put it back together
        # again.  I just want to show you its contents.
        
        x = location_pair[1]
        # ask for element 1 of the variable 'location_pair'
        
        y = location_pair[0]
        # ask for element 0 of the variable 'location_pair'
        
        # notice that they are backwards?  It isn't uncommon
        # to have to do a lot of untangling as you move
        # between coordinate systems.  gmaps expects
        # latitude, longitude (N/S, E/W).  I'd have
        # to double check, but Twitter must've given
        # us longitude, latitude (E/W,N/S).
        
        # To make things more fun:  There are several
        # different ways of reporting longitude/latitude;
        # if your maps look odd, it is probably because
        # you're giving coordinates in the wrong system.
        
        two_in_one = (x,y)
        # Now I have reunited X and Y in a tuple:  that's
        # what those parentheses are for.  If I left
        # out the parentheses, Python would throw an error.
        # The parentheses mean "count this as one value
        # for now."
        
        print(x,y)
        # Just show me the coordinates so I know things are working.
        
        if (x,y) is not None:
            # Finally:  This is just a failsafe.  I had checked earlier
            # to see if 'coordinates' had a value, but I never actually
            # checked to see that X and Y had values.  Occasionally,
            # they didn't, so Python threw an error.  I've fixed that
            # problem at the last second here by saying "As long as 
            # x and y really do have values, add those values to my
            # map's list."  If they don't have values, no problem:  We
            # just skip this last line and start looking for a new 
            # location....
            
            tweet_origins.append(two_in_one)
            
            # alongside our origins, we'll also keep adding
            # the text of each tweet that had an x and y.
            # so tweet_origins[56] will be the xy of
            # tweet_text[56].  Remember too that
            # this_Tweet is where we temporarily iterate
            # through each tweet we're reading.  We
            # can get the text of that tweet by saying
            # this_Tweet['text'], OR we can say
            # this_Tweet.text
            
            tweet_text.append(this_Tweet['text'])
            
            # BREAK BREAK BREAK
            # note that I'm doing this differently from
            # this point forward... so I'm not 
            # really using the arrays I built above...
            
            temp_one = this_Tweet['text']
            temp_two = (x,y)
            temp_three = this_Tweet['user']['name']
            
            # store them together
            
            tweet_total.append({'nom': temp_three, 'txt': temp_one,'geo': temp_two})
41.89736357 -84.06229842
41.50099454 -90.54833265
43.02997657 -87.90996909
40.7913 -77.8587
36.09324636 -79.8878887
34.37694787 -80.06783
28.03164334 -81.94582474
43.16221389 -77.62935278
44.1645 -93.9938
33.64075563 -84.44279574
42.24213283 -97.01351667
43.20944427 -77.95118229
28.1394 -82.462
38.6397 -90.4587
28.15252 -82.35254
44.948232 -93.105406
44.948232 -93.105406
39.94760694 -76.72854643
40.8078265 -73.79745684
39.28396979 -76.62152957
34.59134058 -94.21967159
40.7913 -77.8587
40.36187875 -75.91145676
37.79620872 -122.26251699
40.15501 -80.272935
39.18572 -96.57482
30.6014 -96.3145
32.0289398 -81.1101603
42.71762791 -73.7509919
37.84085319 -122.10908547
37.26428542 -122.01034908
42.10192832 -72.55727359
41.72082388 -73.9356982
43.11575055 -77.51151616
38.9057687 -104.8165838
37.26428542 -122.01034908
42.451108 -79.339659
40.80007 -77.88751
37.271793 -76.709366
37.24261 -78.45986
38.9875 -76.94
42.34705943 -71.08705233
46.85602961 -96.8519007
31.48198744 -83.52733208
35.8489 -90.6672
41.72082388 -73.9356982
34.18479953 -118.57567482
37.24261 -78.45986
37.24261 -78.45986
30.6014 -96.3145
40.95152541 -74.08855448
28.5686 -81.3894
39.56391734 -76.28684767
39.330544 -82.10128
29.5813408 -95.2029037
33.64145002 -84.44211279
42.33604817 -71.16823326
40.51878338 -85.6637898
39.155355 -81.6670729
In [45]:
print(tweet_total)
[{'nom': 'Bonnie Lynch', 'txt': '“you’re so extra” @zkrug19 @ Adrian College https://t.co/JGkaNU11fV', 'geo': (41.89736357, -84.06229842)}, {'nom': 'Haley Grunewald', 'txt': 'So in love with our little Phi Rho fam❤️🌼 @ Augustana College -… https://t.co/qgHIoEAAT9', 'geo': (41.50099454, -90.54833265)}, {'nom': 'Kat Schleicher', 'txt': 'Magic Duo ⚡️👫⚡️Shot in the studio 6 months ago with my college graduation present. I’m slow with… https://t.co/FlQPVsJVLS', 'geo': (43.02997657, -87.90996909)}, {'nom': 'OneShiffSon', 'txt': 'templeofthedog \nA reminder... @ State College, Pennsylvania https://t.co/MWaywTRFao', 'geo': (40.7913, -77.8587)}, {'nom': 'Darren Brand', 'txt': 'Shirt ( @pbjclothingco ) &amp; Showtime .... @ Guilford College https://t.co/5TEHrcImV3', 'geo': (36.09324636, -79.8878887)}, {'nom': 'Bri 🔅', 'txt': 'no better feelin @ Coker College https://t.co/FdIJjltc75', 'geo': (34.37694787, -80.06783)}, {'nom': 'Vicki Lucas', 'txt': 'Just posted a photo @ Florida Southern College https://t.co/nrOS3jl4ZN', 'geo': (28.03164334, -81.94582474)}, {'nom': 'Kimberly Gaggiano', 'txt': 'Rain, sleet, hail, snow, WIND and the GPS College Showcase… https://t.co/c3v39bzrtb', 'geo': (43.16221389, -77.62935278)}, {'nom': 'Myles Schwartz', 'txt': 'cheers to finally being college bffs like we said we were going to be a year ago🌸🌼🌻🌺ily @… https://t.co/Pmn8kzUjjM', 'geo': (44.1645, -93.9938)}, {'nom': 'Alicia Bourda', 'txt': 'Got to see at least one of my college friends while here in Atlanta.… https://t.co/i0wyCbIvxS', 'geo': (33.64075563, -84.44279574)}, {'nom': 'Connor Miller', 'txt': '🐎 @ Wayne State College https://t.co/xGz6n98l8g', 'geo': (42.24213283, -97.01351667)}, {'nom': 'alyssa klock', 'txt': 'my heart &amp; soul @ The College at Brockport State University of New York https://t.co/GMMBcGg3Hb', 'geo': (43.20944427, -77.95118229)}, {'nom': 'Allan Monteclaro', 'txt': 'Schooling my son on some old school arcade game! I used to play this in college after school! I… https://t.co/8SQfeBGjMt', 'geo': (28.1394, -82.462)}, {'nom': "Vito's in the Valley", 'txt': 'Fun Day at The Taste of CBC! @ Christian Brothers College High School https://t.co/rwDr6SCgsC', 'geo': (38.6397, -90.4587)}, {'nom': 'Brian Brown', 'txt': 'Introducing our College LifeGroup to #SnowRolls @suzanne0106 and I always enjoy our time with… https://t.co/PNA6R1EdHR', 'geo': (28.15252, -82.35254)}, {'nom': 'LegiScan MN', 'txt': 'SF3888 [NEW] Saint Paul College asset preservation bond issue and appropriation https://t.co/DsV0eaMCpb', 'geo': (44.948232, -93.105406)}, {'nom': 'LegiScan MN', 'txt': 'SF3887 [NEW] Saint Paul College academic excellence renovation and renewal project design bond issue and appropr... https://t.co/WaBMR2j5SP', 'geo': (44.948232, -93.105406)}, {'nom': 'Mathilde Govers', 'txt': 'these two randos photobombed my pic #ugh @ York College of Pennsylvania https://t.co/ppZiw5x1W8', 'geo': (39.94760694, -76.72854643)}, {'nom': 'NY Maritime Football', 'txt': 'Great work tonight Privateers!! #EarnIt @ SUNY Maritime College https://t.co/vb3UC6JXLN', 'geo': (40.8078265, -73.79745684)}, {'nom': 'John Spear', 'txt': 'First day at mdcrabcrawl college tour included a tour of (and reception at) #camdenyards.… https://t.co/AW8bPqy04j', 'geo': (39.28396979, -76.62152957)}, {'nom': 'Christopher Brown', 'txt': "I'm at Rich Mountain Community College in Mena, AR https://t.co/3M6CpTKIKm", 'geo': (34.59134058, -94.21967159)}, {'nom': 'dani', 'txt': 'Saturday’s with my girls &gt;&gt; @ State College, Pennsylvania https://t.co/If4vk4J9qY', 'geo': (40.7913, -77.8587)}, {'nom': 'Bryan Brubaker', 'txt': 'Details. @ Albright College https://t.co/Z3OxibKdMT', 'geo': (40.36187875, -75.91145676)}, {'nom': 'IG: Dj.Slowpoke', 'txt': 'Mood. @ Laney College https://t.co/GvggpSH7K7', 'geo': (37.79620872, -122.26251699)}, {'nom': 'Kiley Austin', 'txt': 'all bundled up @ Washington &amp; Jefferson College https://t.co/3ub0nQASic', 'geo': (40.15501, -80.272935)}, {'nom': 'Ari Nagaro✨', 'txt': 'I love college - remix weekend 🍹🍸🍺 @ Aggieville https://t.co/SB0YlKFwPp', 'geo': (39.18572, -96.57482)}, {'nom': 'Erica Bonham', 'txt': 'Good friends are like the chili at chilifest: hard to find @ College Station, Texas https://t.co/lfmTiDwTmp', 'geo': (30.6014, -96.3145)}, {'nom': 'LINEUP', 'txt': 'The newest members of the ΔΑ Chapter of Zeta Phi Beta Sorority Inc. SPRING 18’ @ Texas College… https://t.co/48EWObtQhG', 'geo': (32.0289398, -81.1101603)}, {'nom': 'Maddie', 'txt': "Weekend please don't leave us😩 @ Siena College https://t.co/HwAi5zB7fA", 'geo': (42.71762791, -73.7509919)}, {'nom': 'Anne M. Carpenter', 'txt': "Welcoming our new and fully initiated Catholics! @ Saint Mary's College of California https://t.co/0LisOt2eUh", 'geo': (37.84085319, -122.10908547)}, {'nom': 'Alyssa Bilotti', 'txt': 'Still onset !!.\n#teenactress #filming #commercial #bayareatalent #working @ West Valley College https://t.co/HO79CgjCzr', 'geo': (37.26428542, -122.01034908)}, {'nom': 'Victoria Allman', 'txt': 'Retweet. @ Springfield College https://t.co/ovsRnzJKN5', 'geo': (42.10192832, -72.55727359)}, {'nom': 'Caroline', 'txt': 'Will be joining the fam at Marist❤️🦊 #Marist2022 @ Marist College https://t.co/yVMJQ5wxQD', 'geo': (41.72082388, -73.9356982)}, {'nom': 'Chef Carl', 'txt': 'Sunsets in Springtime #sjfc @ St. John Fisher College https://t.co/R5RmpVtyp9', 'geo': (43.11575055, -77.51151616)}, {'nom': 'Dr. Sandy Ho', 'txt': 'Dr. Brian Burnett speaking at the 10year celebration for the College of Education —Leadership,… https://t.co/CyyN2kTlRU', 'geo': (38.9057687, -104.8165838)}, {'nom': 'Ewell Collins', 'txt': '@ibjjf thanks for having me. @ West Valley College https://t.co/FKh5KDfkN5', 'geo': (37.26428542, -122.01034908)}, {'nom': 'Travis', 'txt': 'Just posted a photo @ Suny College At Fredonia https://t.co/jSpgMCSBN7', 'geo': (42.451108, -79.339659)}, {'nom': 'Hayden Christ', 'txt': 'No games in 207 @ The Park at State College https://t.co/PMhoIgvucM', 'geo': (40.80007, -77.88751)}, {'nom': 'Haley Riley', 'txt': 'just me, william, &amp; mary @ College of William and Mary https://t.co/OMouC4N2Gx', 'geo': (37.271793, -76.709366)}, {'nom': 'Leigha Bruce', 'txt': 'We had ourselves a weekend #GreykWek @ Hampden-Sydney College https://t.co/DWAwAUm8sb', 'geo': (37.24261, -78.45986)}, {'nom': 'Eric Owusu', 'txt': 'We go on great dates. #SaraGotCourted @ University of Maryland, College Park https://t.co/W2ckX5Fo0T', 'geo': (38.9875, -76.94)}, {'nom': 'Vlade Guigni', 'txt': 'On the making... Practice time. @ Berklee College of Music https://t.co/QzVGmvQYvN', 'geo': (42.34705943, -71.08705233)}, {'nom': 'Russ Peterson', 'txt': 'Concordia College jazz ensemble I with the one and only Bernie Dressel on drums! @ Delta Hotels… https://t.co/uIOHqzYg4H', 'geo': (46.85602961, -96.8519007)}, {'nom': 'KingHitt', 'txt': 'Happy birthday 😛 @ Abraham Baldwin Agricultural College https://t.co/OTNZlPArcz', 'geo': (31.48198744, -83.52733208)}, {'nom': 'ⓈⒺⓉⒽ', 'txt': 'First college football team to get team issued bath robes...… https://t.co/hhXKTUSEx7', 'geo': (35.8489, -90.6672)}, {'nom': 'Kamryn S', 'txt': 'My first decision as an adult ❤️🦊 #18 @ Marist College https://t.co/rez8ZiqoBi', 'geo': (41.72082388, -73.9356982)}, {'nom': 'Karla Hoffman', 'txt': 'Pre-game huddle. It’s soccer time! ⚽️ Go Olive!  #soccer #soccergirl #soccermom @ Pierce College https://t.co/MrJko1kxO1', 'geo': (34.18479953, -118.57567482)}, {'nom': 'kenzie', 'txt': 'siri, play thunderstruck #rolltigs @ Hampden-Sydney College https://t.co/95DY2Nhki9', 'geo': (37.24261, -78.45986)}, {'nom': 'kenzie', 'txt': 'siri, play thunderstruck #rolltigs @ Hampden-Sydney College https://t.co/95DY2NyV9H', 'geo': (37.24261, -78.45986)}, {'nom': 'lillian sacramento', 'txt': 'didn’t see any chili, but it was a fest💙🎊 @ College Station, Texas https://t.co/ibDxzDzIj6', 'geo': (30.6014, -96.3145)}, {'nom': 'Scottyy Groove💯🗣♿️', 'txt': 'Watch My Moves And Try To Plagiarize Me 💯🤕 #Explorepage @ Bergen Community College https://t.co/hpK0HqxeWG', 'geo': (40.95152541, -74.08855448)}, {'nom': 'Sandra Rizzolo', 'txt': 'What is this archaic book? Had to explain to Natty 😝 @ College Park… https://t.co/nOnhpsnnFk', 'geo': (28.5686, -81.3894)}, {'nom': 'Christine Thomas', 'txt': 'Setting up for performance #2 (and closing night) of Memoirs of Anne. (@ Harford Community College - Joppa Hall in… https://t.co/GtAh974YnS', 'geo': (39.56391734, -76.28684767)}, {'nom': 'matthew slogar', 'txt': 'Thanks for being a college kid again!! Love you @ The J Bar https://t.co/LNamdGjJJ8', 'geo': (39.330544, -82.10128)}, {'nom': 'Dennysav', 'txt': 'Sometimes you just gotta bran life by the horns 🙃🙃🙃😂 @ San Jacinto College South SGA https://t.co/6GOoiLZoql', 'geo': (29.5813408, -95.2029037)}, {'nom': 'seat 1A, please', 'txt': "I'm at United Club in College Park, GA https://t.co/tZoziNm1rD", 'geo': (33.64145002, -84.44211279)}, {'nom': 'Dan Murphy', 'txt': 'Boston College class of ‘22 thank you to my family and friends for… https://t.co/na1mzd9Lt2', 'geo': (42.33604817, -71.16823326)}, {'nom': 'THE ILLUSIONIST', 'txt': 'Surprise visit to see my daughter @ashley_coverly at college.  Love you princess!… https://t.co/n2POMRIbl4', 'geo': (40.51878338, -85.6637898)}, {'nom': 'Weird Hills to Die On', 'txt': 'College Hill (Wood County, WV) https://t.co/BGHZMz9TIC https://t.co/WzbIeDZ2m1', 'geo': (39.155355, -81.6670729)}]

Now we'll add some CSS to allow a box to pop up when we click a marker.

This is actually a bit complicated, so I'll forego a thorough explanation for now: Come see me in my office if you like and we'll rebuild it together.

Suffice it to say that to hold my geo-coded tweets, I've built a huge list that is full of lots of little dictionaries, each of which has two keys: "txt" and "geo". From that, the code below creates a NEW list of locations, grabbing every location from my bigger list; a second list gets built that contains some CSS -- its the stuff in between the triple-double-quotes.

What are Triple Double Quotes? TripleDoubleQuotes are a way to let Python that it should use this as CODE but not try to interpret the code itself -- in this case, because it is CSS, not Python. And those curly braces (around the word 'txt') are pointing out a variable that needs to get filled in when the code actually runs...

See? I told you it wasn't much fun. The result is not really what I wanted to end up with, but it is a start. The text boxes are layered upon one another instead of all just appearing on the topmost layer. And I'd prefer the boxes show up on mouse_over rather than on click (otherwise you have to shut them yourself). But it serves its purpose as a proof of concept in this case.

In [46]:
tweet_locations = [the_tweet['geo'] for the_tweet in tweet_total]
info_box_template="""
<dl><dt>{nom}</dt><dd>{txt}</dd></dl>"""
tweet_data = [info_box_template.format(**the_tweet) for the_tweet in tweet_total]
In [47]:
markers = gmaps.marker_layer(tweet_locations, info_box_content=tweet_data)
myMap.add_layer(markers)
myMap