A good way to delete comments from any programming language

delete_comments.md

A good way to delete comments from any programming language

I find a good way to delete comments from any programming language.

While using bat command, I found it can highlight the code. It was using a library called syntect in rust. But I'm not familiar with rust. So I decided to find a python highlight library. Finally, I found pygments.

It is simple to use and it can delete comments from any programming language easily. Take Python code as an example:

# Parsing source code to Token stream. Ofcourse, you can use other language's lexer.
from pygments.lexers import PythonLexer

# Judge if the token is a comment.
from pygments.token import Token, is_token_subtype

def delete_comments(fname):
    src = open(fname, "r").read()
    dst = open(fname, "w")
    # If Token are comment, you can ignore it and don't write it to dst file.
    for token in PythonLexer().get_tokens(src):
        if not is_token_subtype(token[0], Token.Comment) and token[0] != Token.Literal.String.Doc):
            dst.write(token[1])
            continue
        
        if token[0] == Token.Literal.String.Doc:
            linefeed = src.find("\n", token[1])
            for _ in range(linefeed):
                dst.write("\n")
            continue

test.py before:

#!/bin/python
# coding: utf-8
# import sys
import os
"""
print hello world!
"""
def test(): # this is a function
    s = 'he' + 'll\
o ' + 'wo\
rld!'
    os.system("echo %s###"%s)


#  main function
if __name__ == '__main__':
    test()

test.py after using delete_comments("test.py")

#!/bin/python


import os



def test(): 
    s = 'he' + 'll\
o ' + 'wo\
rld!'
    os.system("echo %s###"%s)



if __name__ == '__main__':
    test()