Overview

toolbox-pyspark

Introduction

The purpose of this package is to provide some helper files/functions/classes for generic PySpark processes.

Key URLs

For reference, these URL's are used:

Type	Source	URL
Git Repo	GitHub	https://github.com/data-science-extensions/toolbox-pyspark
Python Package	PyPI	https://pypi.org/project/toolbox-pyspark
Package Docs	Pages	https://data-science-extensions.com/toolbox-pyspark

Installation

You can install and use this package multiple ways by using any of your preferred methods: pip, pipenv, poetry, or uv.

Using pip:

1. In your terminal, run:

   python3 -m pip install --upgrade pip
   python3 -m pip install toolbox-pyspark
   

2. Or, in your requirements.txt file, add:

   toolbox-pyspark
   

   Then run:

   python3 -m pip install --upgrade pip
   python3 -m pip install --requirement=requirements.txt
   

Using pipenv:

1. Install using environment variables:

   In your Pipfile file, add:

   [[source]]
   url = "https://pypi.org/simple"
   verify_ssl = false
   name = "pypi"
   
   [packages]
   toolbox-pyspark = "*"
   

   Then run:

   python3 -m pip install pipenv
   python3 -m pipenv install --verbose --skip-lock --categories=root index=pypi toolbox-pyspark
   

2. Or, in your requirements.txt file, add:

   toolbox-pyspark
   

   Then run:

   python3 -m run pipenv install --verbose --skip-lock --requirements=requirements.txt
   

3. Or just run this:

   python3 -m pipenv install --verbose --skip-lock toolbox-pyspark
   

Using poetry:

1. In your pyproject.toml file, add:

   [project]
   dependencies = [
       "toolbox-pyspark==1.*",
   ]
   

   Then run:

   poetry sync
   poetry install
   

2. Or just run this:

   poetry add "toolbox-pyspark==1.*"
   poetry sync
   poetry install
   

Using uv:

1. In your pyproject.toml file, add:

   [project]
   dependencies = [
       "toolbox-pyspark==1.*",
   ]
   

Then run:

uv sync

1. Or run this:

   uv add "toolbox-pyspark==1.*"
   uv sync
   

2. Or just run this:

   uv pip install "toolbox-pyspark==1.*"
   

Contribution

Check the CONTRIBUTING.md file or Contributing page.