- Use the Extensions API
Python Modules Available to Indexing Pipeline Extensions
A large number of modules are available for import in indexing pipeline extension (IPE) scripts. The following Python 3 modules may be especially useful:
This list is maintained manually, and might therefore be slightly out of date (e.g., new modules may have been added, or existing modules may have been updated to a newer version since the last edit). Last updated: October 2019.
Validate the complete list of modules and their versions programmatically (see Listing Available Modules Programmatically).
Modules from the Python 3 Standard Library.
beautifulsoup4(4.6.0) - A Python library for pulling data out of HTML and XML files (see beautifulsoup4).
boto3(1.9.0) - Amazon Web Services (AWS) Software Development Kit (SDK) for Python (see boto3).
cryptography(2.7) - A cryptographic library (see cryptography).
python-dateutil(2.8.0) - Provides powerful extensions to the datetime module (see python-dateutil).
requests(2.22.0) - HTTP library for Python (see requests).
urllib3(1.25.5) - An HTTP client (see urllib3).
Listing Available Modules Programmatically
To get a log message containing the current, exhaustive, up-to-date list of all Python 3 modules/versions available for import in any given Python 3 indexing pipeline extension script, you can run a Python 3 indexing pipeline extension that has the following script:
import pkg_resources modules = pkg_resources.working_set modules_list = sorted(["%s, version %s" % (i.project_name, i.version) for i in modules]) log(str(modules_list))
Getting the Python Version Programmatically
To get a log message containing the current Python version used for indexing pipeline extensions, you can run or test an indexing pipeline extension that has the following script:
import sys myPythonVersion = sys.version_info log(str(myPythonVersion))