"Bug Report: Pylint Warning W0102 - Dangerous Default Value in download_pages Function"

[mallick-rebal]

Hello,

I am reaching out regarding your source code file for your Python codes (crawl.py). After running tests using Pylint a few errors present in the source code were found. I felt that it could be something you could fix or look into.

lazynlp/crawl.py:173:0: W0102: Dangerous default value [] as argument (dangerous-default-value)
lazynlp/crawl.py:173:0: W0102: Dangerous default value [] as argument (dangerous-default-value)
lazynlp/crawl.py:222:8: W0105: String statement has no effect (pointless-string-statement)

outputLint.txt

Possible Fix is:

def download_pages(link_file,
                   folder,
                   timeout=30,
                   default_skip=True,
                   extensions=None,
                   domains=None):
    """
    Your function documentation here.
    """
    # Check if extensions and domains are None, and if so, initialize them to empty lists
    if extensions is None:
        extensions = []
    if domains is None:
        domains = []

    # Your function code continues...

This modification ensures that each call to download_pages() gets its own separate empty list for extensions and domains.

I have shown a few errors that I found using Pylint. I also added a link to a text file in which the errors present in all source files were reported using Pylint. Hope this helps.

Regards,
Rebal