BAN-B310 @ 44bb3cc • cybermonkj / qatest

Run summary

a year ago

9e4d9dc..44bb3cc

13 s

Audit required: Use of an insecure method method from urllib detected BAN-B310

Security

Major

12 occurrences in this check

a10 cwe-918 owasp top 10

Audit url open for permitted schemes. Allowing use of file:/ or custom schemes is often unexpected.

Test.py

608        # Api call to read data for northumbria
609        with urllib.request.urlopen(northumbriaUrl_04_2021) as resp1:
610            northumbria2021Data = resp1.read()
611        with urllib.request.urlopen(northumbriaUrl_04_2020) as resp2:612            northumbria2020Data = resp2.read()
613            # Api call to read data for cleveland
614

Audit url open for permitted schemes. Allowing use of file:/ or custom schemes is often unexpected.

Test.py

606        northumbriaUrl_04_2021 = getUrl("northumbria", "2021-03")
607        northumbriaUrl_04_2020 = getUrl("northumbria", "2020-04")
608        # Api call to read data for northumbria
609        with urllib.request.urlopen(northumbriaUrl_04_2021) as resp1:610            northumbria2021Data = resp1.read()
611        with urllib.request.urlopen(northumbriaUrl_04_2020) as resp2:
612            northumbria2020Data = resp2.read()

Audit url open for permitted schemes. Allowing use of file:/ or custom schemes is often unexpected.

Test.py

553        # Api call to read data for northumbria
554        with urllib.request.urlopen(northumbriaUrl_04_2021) as resp1:
555            northumbria2021Data = resp1.read()
556        with urllib.request.urlopen(northumbriaUrl_04_2020) as resp2:557            northumbria2020Data = resp2.read()
558            # Api call to read data for cleveland
559

Audit url open for permitted schemes. Allowing use of file:/ or custom schemes is often unexpected.

Test.py

551        # clevelandUrl_07_2021   = getUrl("cleveland", "2021-07")
552        # clevelandUrl_08_2021   = getUrl("cleveland", "2021-08")
553        # Api call to read data for northumbria
554        with urllib.request.urlopen(northumbriaUrl_04_2021) as resp1:555            northumbria2021Data = resp1.read()
556        with urllib.request.urlopen(northumbriaUrl_04_2020) as resp2:
557            northumbria2020Data = resp2.read()

Audit url open for permitted schemes. Allowing use of file:/ or custom schemes is often unexpected.

Test.py

467            clevelandJuneData = resp4.read()
468        with urllib.request.urlopen(clevelandUrl_07_2021) as resp5:
469            clevelandJulyData = resp5.read()
470        with urllib.request.urlopen(clevelandUrl_08_2021) as resp6:471            clevelandAugustData = resp6.read()
472        # Read and load data in panada dataframe
473        json_format_cleveland_june = json.loads(clevelandJuneData)

Audit url open for permitted schemes. Allowing use of file:/ or custom schemes is often unexpected.

Test.py

465        # Api call to read data for cleveland
466        with urllib.request.urlopen(clevelandUrl_06_2021) as resp4:
467            clevelandJuneData = resp4.read()
468        with urllib.request.urlopen(clevelandUrl_07_2021) as resp5:469            clevelandJulyData = resp5.read()
470        with urllib.request.urlopen(clevelandUrl_08_2021) as resp6:
471            clevelandAugustData = resp6.read()

Audit url open for permitted schemes. Allowing use of file:/ or custom schemes is often unexpected.

Test.py

463        with urllib.request.urlopen(northumbriaUrl_08_2021) as resp3:
464            northumbriaAugustData = resp3.read()
465        # Api call to read data for cleveland
466        with urllib.request.urlopen(clevelandUrl_06_2021) as resp4:467            clevelandJuneData = resp4.read()
468        with urllib.request.urlopen(clevelandUrl_07_2021) as resp5:
469            clevelandJulyData = resp5.read()

Audit url open for permitted schemes. Allowing use of file:/ or custom schemes is often unexpected.

Test.py

460            northumbriaJuneData = resp1.read()
461        with urllib.request.urlopen(northumbriaUrl_07_2021) as resp2:
462            northumbriaJulyData = resp2.read()
463        with urllib.request.urlopen(northumbriaUrl_08_2021) as resp3:464            northumbriaAugustData = resp3.read()
465        # Api call to read data for cleveland
466        with urllib.request.urlopen(clevelandUrl_06_2021) as resp4:

Audit url open for permitted schemes. Allowing use of file:/ or custom schemes is often unexpected.

Test.py

458        # Api call to read data for northumbria
459        with urllib.request.urlopen(northumbriaUrl_06_2021) as resp1:
460            northumbriaJuneData = resp1.read()
461        with urllib.request.urlopen(northumbriaUrl_07_2021) as resp2:462            northumbriaJulyData = resp2.read()
463        with urllib.request.urlopen(northumbriaUrl_08_2021) as resp3:
464            northumbriaAugustData = resp3.read()

Audit url open for permitted schemes. Allowing use of file:/ or custom schemes is often unexpected.

Test.py

456        clevelandUrl_07_2021 = getUrl("cleveland", "2021-07")
457        clevelandUrl_08_2021 = getUrl("cleveland", "2021-08")
458        # Api call to read data for northumbria
459        with urllib.request.urlopen(northumbriaUrl_06_2021) as resp1:460            northumbriaJuneData = resp1.read()
461        with urllib.request.urlopen(northumbriaUrl_07_2021) as resp2:
462            northumbriaJulyData = resp2.read()

Audit url open for permitted schemes. Allowing use of file:/ or custom schemes is often unexpected.

Test.py

411        )
412        with urllib.request.urlopen(clevelandUrl) as resp1:
413            clevelandData = resp1.read()
414        with urllib.request.urlopen(northumbriaUrl) as resp2:415            northumbriaData = resp2.read()
416        # Read and load data in panada dataframe
417        jsonFormatCleveland = json.loads(clevelandData)

Audit url open for permitted schemes. Allowing use of file:/ or custom schemes is often unexpected.

Test.py

409        clevelandUrl = (
410            "https://data.police.uk/api/stops-force?force=cleveland&date=2021-06"
411        )
412        with urllib.request.urlopen(clevelandUrl) as resp1:413            clevelandData = resp1.read()
414        with urllib.request.urlopen(northumbriaUrl) as resp2:
415            northumbriaData = resp2.read()

Description

urllib not only opens http:// or https:// URLs, but also ftp:// and file://. With this, it might be possible to open local files on the executing machine which might be a security risk if the URL to open can be manipulated by an external user.

The urllib.request module defines functions and classes which help in opening URLs. urllib.request.open can open ftp:// and file:// URLs. This is usually not intended and makes the application vulnerable to Server Side Request Forgery attack. Performing requests from user-provided data could allow attackers to make requests on the internal network or change, retrieve or delete sensitive information. You are yourself responsible for validating the URL before opening it with urllib.

It is recommended to validate the user-provided data, such as the URL and headers used to construct the request.

Since this is an audit issue, some occurrences may be harmless here. The goal is to bring the issue in attention. Please make sure that the url is trusted. If the occurrences doesn't seem to be valid, please feel free to ignore them.

Bad practice

req = urllib.Request.request(url)
resp = urllib.request.urlopen(req)

# Validate URL before opening it
if url.lower().startswith('http'):
  req = urllib.Request.request(url)
else:
  raise ValueError from None

with urllib.request.urlopen(req) as resp:
  [...]

## References:
- OWASP Top 10 2021 Category A10 - [Server Side Request Forgery](https://owasp.org/Top10/A10_2021-Server-Side_Request_Forgery_%28SSRF%29/)
- [CWE-918](https://cwe.mitre.org/data/definitions/918.html) - Server-Side Request Forgery (SSRF)

cybermonkj / qatest

Bad practice

Recommended