Make America Tweet Again Part 3

Part 3 | Automating Our Program

A Tutorial on Scheduling Twitter Processes with Advanced Python Scheduler.

This is a continuation of  Part 2 of Make America Tweet Again.  In Part 2 we created a Markov Chain style system of generating random Trump-like Tweets.   Source code can be found on my GitHub Page for MakeAmericaTweetAgain.


What Your Project Folder Should Look Like

  1. env/:  This is the directory for our Virtual Environment Files
  2. requirements.txt:  These are the installed modules to our environment
  3. twitterKeys.py:  This is the Python module used for holding our Twitter Keys.
  4. api.py:  A small module we made to give us an authenticated Twitter API
  5. getTweetsFromUser.py:  A program that takes in a command line argument for a Twitter Screen Name and extracts tweets from their timeline.
  6. tweetGenerator.py:  A program that holds our class for generation of tweets and generation of Trump Specific Tweets.
  7. Any Output from Files Listed Above:  In this tutorial, we’ve saved a lot of JSON objects to hold a variety of twitter and tweet generation data.  Depending on what you saved and how you tested your program, you are likely to have such artifacts in your folder.

The General Idea

A summary of our program:

  1. Gather a new set of Tweets every day.  Let’s say this will happen every night at 11:59PM.
  2. Tweet every 15 minutes from the hours of 8:00AM to 9:00PM
  3. Get a list of people that are following us and follow them back.
  4. If somebody Retweets one of our Tweets, we will like it and follow the poster.
  5. Reply to and like a selection of Tweets that contain the phrase “MakeAmericaGreatAgain”

People reading this tutorial are likely on a variety of systems.  I’ll focus on creating a program that can be run from a bash terminal, but at the end, I’ll touch upon how you might upload your program to an external server or cloud service to run in the wild.


APScheduler Setup and Introduction

We will be using the APScheduler library to implement our time-dependent functionality.  To install the library we must enter our Virtual Environment and install via Pip.

I’ll walk us through some quick examples to show us how to set up some timed tasks.  Open your Python shell:

This is a pretty straightforward example of how the scheduler works.  You import the scheduler object.  After that you add an instance of the object.  Once you have your instance, you define a function of your choosing.  The  add_job() function here adds your defined function, and repeats it at an interval of 10 seconds.  You should see your function executing in your shell at the defined interval.

So now we have a rough idea of how our scheduling system will work and we have some goals for our program.  Let’s start coding up our scheduling system.


Outline of Our Program

Create a file in our directory called  tweetScheduler.py and add the following code:

You might notice a few new things in this code.  First of all, I’m adding in some helper functions to make our functionality simpler. rateLimitNotExceeded() and cleanTweet().

  • rateLimitNotExceeded():  If we max out our API calls to Twitter our program will not work, we can use this function to make sure we haven’t exceeded the limits.
  • cleanTweet():  When I was testing the program two tiny details really bothered me.  We are given HTML escape coding in our Tweet data.  Trump uses the character “&” quite a bit, and our tweet set will contain “&” instead.  In addition, often times Trump will quote somebody with punctuation in his tweets.  It is possible to start a quote and not end with one given the way or Markov Chains are generated.  We can also get a word that ended a quote without ever starting one.  So this method will also help us remove instances of single quotation marks.  Instead of editing our existing code, we’ll just add a quick function to clear up these issues.

We also use the function sleep() to enter an infinite loop.  The loop just tells the computer to wait for 60 seconds before looping again to keep our program open and allow our scheduler to keep operating.

At the bottom of our code, we add all the jobs to our scheduler.  The only difference between interval and cron arguments for the add_job() function is that interval repeats on the interval and cron repeats on a given time of day.

The FOLLOWERS variable just allows us to store a list of followers.  We do this because API calls are costly and we want to minimize them when we can.  If we get a list of all the people following our bot, and follow them back, we can store their id’s in this list.  So the next time we get our followers, we don’t attempt to follow them a second time and save ourselves another API call.


Some Initial Groundwork

Edit the code in tweetScheduler.py to replace everything above the function updateTweets() with the following code:

This code will take care of our set up for us.  I’ll walk through the main points of the code below.

  1. Imports:  We import system, datetime.   system gives us a way to run our getTweetsFromUser code we created in Part1 within our program.   datetime gives us a time stamp that we’ll print out whenever our jobs run to make sure everything’s running smoothly.  We also import our getAPI to gain access to Tweepy and the Twitter API, and
  2. Print Statements:  We include these for observation’s sake when running our code.  We can also redirect the output into a text file for later inspection if we so choose.
  3. Try/Excepts:  We include these to handle any unexpected errors.  If our API fails or any other error occurs the program will exit gracefully.  The first is a general exception to handle API errors.  The nested statement either will open up an existing Tweet set if needed or gather a new one like in Part 1.

Function Definitions

The heavy lifting of this file is done by the scheduled jobs.  Add these function definitions to tweetScheduler.py:

A quick summary of the functions is as follows:

  • updateTweets():  This function should look familiar.  Basically, we are running our code from Part1 with the os.system module.  It will gather a new set of Tweets and save them.  The try/except statement will just ensure that the program doesn’t crash if the connection went out or the API is maxed out.
  • postTweet():  The datetime.now() gives us the time.  We can inspect the hour member field to make sure we are in the time window we want to send out Tweets.  We get a Tweet from our TrumpTweetGenerator object and clean it.  Then we just print some output to know all is going well and use the API to post this tweet to our account.
  • friendFollowers():  First we collect the people following us from the API.  For each of those people, we create a friendship through the API if we haven’t reached our API limit.  We use FOLLOWERS to keep track of who we already follow so we don’t waste a bunch of API calls trying to make connections with the same set of people over and over.
  • likeAndReplyToMAGA():  Again, we use the API to search for “MakeAmericaGreatAgain.”  Once we have a list of tweets it’s easy to use it to like and reply to that specific Tweet.
  • likeAndFollowRetweets():  The API is called again to get the list of people who have Retweeted our bot.  We get the list of these tweets and reply to and like them in a loop.  Again, we reference FOLLOWERS to make sure we’re not being redundant.
  • rateLimitNotExeeded():  The Tweepy API provides a great method to see if we still have calls left.  It costs no API calls to check this number, so we use it liberally.
  • cleanTweet():  Simply put, it looks for single quotation marks or the HTML escape code for “&”.  We just remove this junk data and move on.

Running Your Program

There are many options for hosting a program in the cloud.  Many of these are beyond the scope of this article.  I myself, have been testing this program running on a Digital Ocean server.  However, running this from your command line should suffice to show you how the program can work in action.  Make sure you’re inside your virtual environment and type the following command in your Terminal:

That’s it!  You should see some of the print statements executing.  The following commands will allow the program to continue execution even if you want to use the Terminal to do something else, or you want to close your shell session:

Note the first command is Ctrl + z.  Now you’re free to do anything and your computer should still execute the commands if it remains on or is sleeping.


Final Notes

There is a lot of room for improvement for this project.  We could spend a lot more time refining our text generation.  Now it might be a garbage sentence, consist of only a few words, or contain other misplaced characters.  We might want to take the time to create a Linux startup service to run off a cloud server to be running constantly over time.  Perhaps we want to more carefully configure our bot to gather more followers, and engage in other marketing activities.  For now, I’ll leave these tasks to the reader.

I hope you’ve enjoyed this Tutorial.  If you have any other topics you’d like to read about, questions, or would like to share some output of your bot, share it in the comments below.

Thanks!

REFERENCES

One thought on “Make America Tweet Again Part 3

Leave a Reply

Your email address will not be published. Required fields are marked *