pickle module BAN-B301361 self.log_count = 0
362 try:
363 with open(self.datfilename, 'rb') as datfile:
364 self.history_dict = pickle.load(datfile)365 except (OSError, EOFError):
366 # no saved history exists yet, or history dump broken
367 self.history_dict = {}133 def _load_cache(self):
134 """Load the cache entry."""
135 with self._cachefile_path().open('rb') as f:
136 self.key, self._data, self._cachetime = pickle.load(f)137 return True
138
139 def parse_key(self):299 self.datafile = pywikibot.config.datafilepath(
300 f'{self.site.dbName()}-catmovebot-data')
301 with suppress(OSError), open(self.datafile, 'rb') as inp:
302 self.record = pickle.load(inp)303 if self.record:
304 with open(self.datafile + '.bak', 'wb') as f:
305 pickle.dump(self.record, f, protocol=config.pickle_protocol) 355 pywikibot.info('Reading dump from '
356 + config.shortpath(self.filename))
357 with open_archive(self.filename, 'rb') as f:
358 databases = pickle.load(f) 359 # keys are categories, values are 2-tuples with lists as
360 # entries.
361 self.cat_content_db = databases['cat_content_db']1244 try:
1245 filename = self._cachefile_path()
1246 with filename.open('rb') as f:
1247 uniquedescr, self._data, self._cachetime = pickle.load(f)1248
1249 if uniquedescr != self._uniquedescriptionstr():
1250 raise RuntimeError('Expected unique description for the cache 'The pickle module is not secure against erroneous or maliciously constructed data. Never unpickle data received from an untrusted or unauthenticated source.
Python's pickle module is used for serializing and de-serializing a Python object structure. Data serialization is the process of converting structured data to a format that allows sharing or storage of the data in a form that allows recovery of its original structure.
Insecure deserialization is when an application deserializes the data that it gets without any kind of validation, or even the authenticity of the data. It is easy to execute arbitrary code when unpickling data. Unpickling can be exploited to execute arbitrary commands on your machine.
If pickle is not absolutely necessary for the use-case, consider using a safer serialization, like PyYaml.PyYAML is a YAML parser and emitter for Python. YAML is language-agnostic and human-readable serialization format. But pickle has its advantages too. Pickle format is specific to Python and can represent a wide variety of data structures and objects where as YAML represents simple data types & structures in a language-portable manner.
Recommended practices when using pickle module:
- It should not be used between unknown parties. Unpickling data from malicious client can cause remote execution attack. Only unpickle data you trust.
- Communicating parties should have an encrypted network connection. Otherwise, data to be unserialized can be modified and thus unsafe to pickle.
- Consider signing data with hmac if you need to ensure that it has not been tampered with. Pickle can be signed before storage or transmission, and its signature can be verified before loading it on the receiver side.
Refer to this blog post to know more about dangers of using pickle module.
Bad practice
from flask import request
import picke
@app.route('/pickle')
def load():
data = request.GET.get("data")
conf = pickle.load(data) # Insecure. Avoid using pickle
Recommended
from flask import request
import yaml
@app.route('/yaml')
def load():
data = request.GET.get("data")
conf = yaml.load(data) # Secure
References:
- OWASP Top 10 2021 Category A08 - Software and Data Integrity Failures
- SANS Top 25
- CWE 502 - Deserialization of Untrusted Data
- pickle module
- yaml module