Merge pull request #31 from ray310/010

010

Author: ray310

.github/workflows/deploy_page.yml

@@ -0,0 +1,28 @@
+name: Deploy Project Site
+on:
+  push:
+    branches:
+      - main
+permissions:
+  contents: write
+jobs:
+  deploy:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - name: Configure Git Credentials
+        run: |
+          git config user.name github-actions[bot]
+          git config user.email 41898282+github-actions[bot]@users.noreply.github.com
+      - uses: actions/setup-python@v5
+        with:
+          python-version: 3.x
+      - run: echo "cache_id=$(date --utc '+%V')" >> $GITHUB_ENV
+      - uses: actions/cache@v4
+        with:
+          key: mkdocs-material-${{ env.cache_id }}
+          path: .cache
+          restore-keys: |
+            mkdocs-material-
+      - run: pip install mkdocs-material mkdocstrings mkdocstrings-python
+      - run: mkdocs gh-deploy --force

.gitignore

@@ -1,19 +1,21 @@
 # file types
 *.docx
 *.env
-*.ipynb
 *.pages
 
 # patterns
 *.egg-info*
+*ipynb*
+conda_environment_dev_*
 
 # folders
 .coverage
 .idea
 data
-dev
+notes
 dist
 htmlcov
+site
 __pycache__
 
 # files

CHANGELOG.md

@@ -1,14 +1,22 @@
 # Changelog
 
 ## Unreleased
+
+## 0.1.1 - Unreleased
 ### Added
 - functionality to detect time series gaps
 
 ____
-## 0.1.0 - Unreleased
+## 0.1.0 - 2024-07-14
 ### Added
-- Split reports module
-- Improved project documentation
+- Add memory usage to `DataFrameProfile` [gh-30](https://github.com/ray310/Panda-Helper/issues/30)
+- Improve formatting of `distribution_stats` function output [gh-29](https://github.com/ray310/Panda-Helper/issues/29)
+- Improved project documentation with project website [gh-2](https://github.com/ray310/Panda-Helper/issues/2)
+
+### Changed
+- [Split reports module into `profiles` and `stats`](https://github.com/ray310/Panda-Helper/commit/93320860834e757ab18d86c2b9334efb05738662)
+- [Renamed `save_report` method to `save`](https://github.com/ray310/Panda-Helper/commit/876c5f5af8906081f96aff1f1f0ba9d5754a719a)
+- [Refactored tests to use pytest fixtures](https://github.com/ray310/Panda-Helper/commit/ff2bf2dd6e73dd4747b62faef4bd350949866a91)
 
 ____
 ## 0.0.4 - 2024-07-09

README.md

@@ -5,232 +5,10 @@
 ![Lint/Format Status](https://github.com/ray310/Panda-Helper/actions/workflows/format_lint.yml/badge.svg)
 
 # Panda-Helper: Quickly and easily inspect data
-Panda-Helper is a simple data-profiling utility for Pandas' DataFrames and Series.
+Panda-Helper is a simple, open-source, Python data-profiling utility for Pandas' DataFrames and Series.
 
 Assess data quality and usefulness with minimal effort.
 
 Quickly perform initial data exploration, _so you can move on to more in-depth analysis_.
 
------
-### DataFrame profiles:
-- Report shape
-- Detect duplicated rows
-- Display series names and data types
-- Calculate distribution statistics on null values per row providing a view on data completeness
-
-__Sample DataFrame profile__<br>
-_Vehicles passing through toll stations_
-
-    DataFrame-Level Info
-    -------------------------  ------------
-    DF Shape                   (1586280, 6)
-    Duplicated Rows             2184
-
-    Column Name                 Data Type
-    --------------------------  -----------
-    Plaza ID                    int64
-    Date                        object
-    Hour                        int64
-    Direction                   object
-    # Vehicles - ETC (E-ZPass)  int64
-    # Vehicles - Cash/VToll     int64
-
-    Summary of Nulls Per Row
-    --------------------------  -----------
-    count                       1.58628e+06
-    min                         0
-    1%                          0
-    5%                          0
-    25%                         0
-    50%                         0
-    75%                         0
-    95%                         0
-    99%                         0
-    max                         0
-    median                      0
-    mean                        0
-    median absolute deviation   0
-    standard deviation          0
-    skew                        0
-
------
-### Series profiles report the:
-- Series data type
-- Count of non-null values in the series
-- Number of unique values
-- Count of null values
-- Counts and frequency of the most and least common values
-- Distribution statistics for numeric-like data
-
-__Sample profile of categorical data__<br>
-_Direction vehicles are traveling_
-
-    Direction Info
-    ----------------  -------
-    Data Type         object
-    Count             1586280
-    Unique Values     2
-    Null Values       0
-
-    Value      Count  % of total
-    -------  -------  ------------
-    I         814100  51.32%
-    O         772180  48.68%
-
-__Sample profile of numeric data__<br>
-_Hourly vehicle counts at tolling points_
-
-    # Vehicles - ETC (E-ZPass) Info
-    ---------------------------------  -------
-    Data Type                          int64
-    Count                              1586280
-    Unique Values                      8987
-    Null Values                        0
-
-      Value    Count  % of total
-    -------  -------  ------------
-          0     3137  0.20%
-         43     1762  0.11%
-         44     1743  0.11%
-         40     1712  0.11%
-         42     1699  0.11%
-         41     1682  0.11%
-         39     1676  0.11%
-         37     1673  0.11%
-         48     1659  0.10%
-         46     1654  0.10%
-         38     1646  0.10%
-         45     1641  0.10%
-         36     1636  0.10%
-         52     1574  0.10%
-         47     1572  0.10%
-         50     1571  0.10%
-         51     1555  0.10%
-         53     1547  0.10%
-         55     1543  0.10%
-         34     1534  0.10%
-       8269        1  0.00%
-       8438        1  0.00%
-       8876        1  0.00%
-       8261        1  0.00%
-       8694        1  0.00%
-
-    Statistic                            Value
-    -------------------------  ---------------
-    count                          1.58628e+06
-    min                            0
-    1%                            25
-    5%                            68
-    25%                          407
-    50%                         1054
-    75%                         2071
-    95%                         3583
-    99%                         6308
-    max                        16854
-    median                      1054
-    mean                        1373.16
-    median absolute deviation    751
-    standard deviation          1253.1
-    skew                           1.69154
-
------
-### Installing Panda-Helper
-`pip install panda-helper`
-
------
-### Using Panda-Helper
-__Profiling a DataFrame__<br>
-Create the DataFrameProfile and then display it or save the profile.
-```python
-import pandas as pd
-import pandahelper as ph
-
-data = {
-    "user_id": [1, 2, 3, 4, 4],
-    "transaction": ["purchase", "return", "purchase", "exchange", "exchange"],
-    "amount": [100.00, None, 1400.00, 85.12, 85.12],
-    "survey": [None, None, None, "online", "online"],
-}
-df = pd.DataFrame(data)
-df_profile = ph.DataFrameProfile(df)
-df_profile
-```
-
-    DataFrame-Level Info
-    -------------------------  ------
-    DF Shape                   (5, 4)
-    Obviously Duplicated Rows  1
-
-    Column Name    Data Type
-    -------------  -----------
-    user_id        int64
-    transaction    object
-    amount         float64
-    survey         object
-
-    Summary of Nulls Per Row
-    --------------------------  --------
-    count                       5
-    min                         0
-    1%                          0
-    5%                          0
-    25%                         0
-    50%                         1
-    75%                         1
-    95%                         1.8
-    99%                         1.96
-    max                         2
-    median                      1
-    mean                        0.8
-    median absolute deviation   1
-    standard deviation          0.83666
-    skew                        0.512241
-
-```python
-df_profile.save_report("df_profile.txt")
-```
-
-__Profiling a Series__<br>
-Create the SeriesProfile and then display it or save it. That's it!
-```python
-series_profile = ph.SeriesProfile(df["amount"])
-series_profile
-```
-    amount Info
-    -------------  -------
-    Data Type      float64
-    Count          4
-    Unique Values  3
-    Null Values    1
-
-      Value    Count  % of total
-    -------  -------  ------------
-      85.12        2  50.00%
-     100           1  25.00%
-    1400           1  25.00%
-
-    Statistic                       Value
-    -------------------------  ----------
-    count                         4
-    min                          85.12
-    1%                           85.12
-    5%                           85.12
-    25%                          85.12
-    50%                          92.56
-    75%                         425
-    95%                        1205
-    99%                        1361
-    max                        1400
-    median                       92.56
-    mean                        417.56
-    median absolute deviation     7.44
-    standard deviation          654.998
-    skew                          1.99931
-
-```python
-series_profile.save_report("amount_profile.txt")
-```
-____
-### Sample data obtained from:
-- https://data.ny.gov/Transportation/Hourly-Traffic-on-Metropolitan-Transportation-Auth/qzve-kjga/data
-- https://data.cityofnewyork.us/Public-Safety/Motor-Vehicle-Collisions-Crashes/h9gi-nx95
+Please see [project website](https://ray310.github.io/Panda-Helper/).

conda_environment_dev.yaml

@@ -10,16 +10,20 @@ dependencies:
     - codespell
     - coverage=7.2.2
     - jupyter=1.0.0
-    - mkdocs
+    - lxml=5.2.1
+    - mkdocs=1.6.0
     - notebook=7.0.8
     - numpy=1.26.4
     - pandas=2.2.2
     - pip=24.0
     - pre-commit=3.4.0
-    - pydocstyle=6.3.0
     - pylint=3.2.2
     - pytest=7.4.4
+    - ruff=0.3.5
     - scipy=1.13.1
     - twine=4.0.2
     - pip:
+        - mkdocs-material==9.5.28
+        - mkdocstrings==0.25.1
+        - mkdocstrings-python==1.10.5
         - tabulate==0.9.0

docs/api.md

@@ -0,0 +1,10 @@
+---
+description: Panda-Helper API Reference. Detailed description of the Panda-Helper API.
+---
+
+# API Reference
+::: pandahelper.profiles
+
+<br>
+
+::: pandahelper.stats

docs/assets/images/panda.png

@@ -0,0 +1,41 @@
+---
+hide:
+  - navigation
+  - toc
+description: Panda-Helper Documentation. Panda-Helper is a simple, Python data-profiling tool for Pandas’ DataFrames and Series that allows you to assess data quality and usefulness with minimal effort.
+---
+
+# Panda-Helper
+___Quickly and easily inspect data so you can move on to more in-depth analysis.___
+
+Panda-Helper is a simple, open-source, Python data-profiling tool for Pandas’ DataFrames and Series
+that allows you to assess data quality and usefulness with minimal effort.
+
+<div class="grid cards" markdown>
+
+-   [:material-hammer-wrench:{ .lg .middle } __Install Panda-Helper__](install.md)
+
+    ---
+
+    Install `pandahelper` with `pip` or `anaconda` and get up
+    and running in minutes
+
+-   [:octicons-book-16:{ .lg .middle} __API Reference__](api.md)
+
+    ---
+
+    Detailed description of the Panda-Helper API
+
+-   [:material-television-guide:{ .lg .middle } __User Guide__](user_guide.md)
+
+    ---
+
+    How to use Panda-Helper with examples
+
+-   [:simple-github:{ .lg .middle } __Source Code__](https://github.com/ray310/Panda-Helper)
+
+    ---
+
+    Review, clone, or fork the source code
+
+</div>