Problem 1 Text Analysis

Create a Python file called analysis.py that will perform text analysis on files. For this question, assume that each space (” “) in the document separates one word from the next – so any use of the term ‘word’ means a string that occurs between two spaces (or in two special cases, between the start of the file and a space, or between a space and the end of the file). You can also assume there is no punctuation or other symbols present in the files – only words separated by spaces. If you want to see examples of the type of text, look in the testfile_.txt files included on cuLearn. You must implement and test the following functions inside of your analysis.py file:

1) load(str) – Takes a single string argument, representing a filename. The program must open the file and parse the text inside. This function should initialize the variables (e.g., lists, dictionaries, other variables) you need to solve the remainder of the problem. This way, the file contents can be parsed once and the functions below can be executed many times without re-reading the file, which is a slow process. This function should also remove any information stored from a previous file when it is called (i.e., you start from nothing every time load is called).

2) commonword(list) – Takes a single list-type argument which contains string values. The function should operate as follows: a. If the list is empty or none of the words specified in the list occur in the text that has been loaded, the function should return None. b. Otherwise, the function should return the word contained in the list that occurs most often in the loaded text – or any one of the most common, in the case of a tie.

3) commonletter(list) – Takes a single list-type argument which contains single character strings (i.e., letters/characters). The function should operate as follows: a. If the list is empty or none of the letters specified in the list occur in the text that has been loaded, the function should return None. COMP 1005/1405 – S17 – A5 Due Tuesday, June 13 at 11:55 PM 2 b. Otherwise, the function should return the letter contained in the list that occurs most often in the loaded text – or any one of the most common, in the case of a tie.

4) commonpair(str) – Takes a single string argument, representing the first word. This function should return the word that most frequently followed the given argument word (or one of, in case of ties). If the argument word does not appear in the text at all, or is never followed by another word (i.e., is the last word in the file), this function should return None.

5) countall() – Returns the total number of words found in the text that has been loaded. That is, the word count of the document.

6) countunique() – Returns the number of unique words in the text that has been loaded. This is different than the previous function, as it should not count the same word more than once. You can use the analysistester.py file from cuLearn, along with the posted text files, to test your functions. You can also create additional text files of your own to further test the correctness of your program. If you want an easy way to find out the necessary information about a file you created, you can copy/paste the contents into the form at http://textalyser.net. It can give you the total word count, unique word count, word frequency to determine the most common word, and common word pairs.