Skip to content

Instantly share code, notes, and snippets.

@Michaelliv
Last active September 15, 2020 20:14
Show Gist options
  • Select an option

  • Save Michaelliv/f8aaae90ec0de55594bbbc46c9aa0e36 to your computer and use it in GitHub Desktop.

Select an option

Save Michaelliv/f8aaae90ec0de55594bbbc46c9aa0e36 to your computer and use it in GitHub Desktop.
def count_terms_in_memory():
total_count: Counter = Counter()
with open(DATA_PATH, encoding="utf8") as f:
# Read the file into memory
lines = f.readlines()
# Lower case each line, split by space
for line in lines:
terms = line.lower().split(' ')
total_count += Counter(terms)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment