0.10.0: All new, and live for 2025. We're now a knowledge bank!

Major reorientation (or "pivot" as the cool kids say) for the project. OgbujiPT is now a general-purpose knowledge bank system for LLM-based applications. It provides a unified API for storing, retrieving, and managing semantic knowledge across multiple backends, with support for dense vector search, sparse retrieval, hybrid search, and more.

As always, we build with Pythonic simplicity and transparency in mind; avoiding the over-frameworks that plague the LLM ecosystem.

Please see the discussion, as this is ongoing work: #92

Even more proper release for PGVector fixes

Deal with the weirdness in building wheels. REALLY needs a proper fix.

Proper release for PGVector fixes

Deal with the weirdness in building wheels. Needs a proper fix.

v0.9.4

Added

• (PGVector) Support for init via DSN connection string: PGVectorHelper.from_conn_string()
• (PGVector) Support setting max & min connection pool size in PGVectorHelper.from_*() methods
• (PGVector) Support for DB schema use for PGVector, e.g., constructors now take a schema kwarg.
• (PGVector) Support for halfvec half-precision vector type & other quantized types, plus HNSW indexing, including half-precision indexing.
• (PGVector) stringify_json flag on PGVector helpers, to help deal with cases where JSONB type registration isn't reliable

Fixed

• llm_wrapper.openai_chat_api call to take messages arg, as expected, not prompt

0.9.3

[0.9.3] - 20240722

Added

• demo/chat_doc_folder.py - "chat my folder of docs" demo
• llm_wrapper.response_type to indicate tool-call LLM responses

Fixed

• llm_wrapper.llm_response object generation

0.9.2

Added

• joiner param to text_helper.text_split() for better control of regex separator handling
• query filter mix-in, embedding.pgvector.match_oneof(), for use with meta_filter argument to DB.search
• llm_wrapper.response_type to differentiate tool calling vs regular LLM responses

Changed

• Index word loom items by their literal default language text, as well
• Cleaned up PGVector query-building logic

Fixed

• llm_wrapper.llm_response objects to handle tool calls
• failure of some matching scenarios in embedding.pgvector.match_exact()

Removed

• Previously deprecated first_choice_text & first_choice_message methods

0.9.1

[0.9.1] - 20240604

Fixed

• Demos & testing around text_helper.text_split_fuzzy()

0.9.0

[0.9.0] - 20240604

Added

• text_helper.text_split_fuzzy() as basically a generator version of text_helper.text_splitter(), and deprecate the latter
• text_helper.text_split() which brooks no overlap
• embedding.pgvector_data module for vector database embedding of data fields using PGVector. Document fields are now treated as simple text, with specialized metadata generalized in the new metadata field
• query filter mix-in, embedding.pgvector.match_exact(), for filtering vector search by metadata fields via new meta_filter argument to DB.search. This restores some of the tag matching search functionality that's been removed.
• clone() method on wordloom.language_item class (formerly text_item)
• meta and preserve_key args on wordloom.language_item (formerly text_item) initializer, to preserve TOML table items and top-level TOML key for each language item, respectively, in object properties

Changed

• Deprecation of text_helper.text_splitter()
• tags field of table encapsulated in embedding.pgvector_data now modified into a general, JSON meta field
• Word Loom now uses _ as the TOML table key for text content. The former text is deprecated
• Word Loom now uses _m as the TOML table key for text substitution markers. The former markers is deprecated
• Word Loom now reserves TOML table keys beginning with _, and passes on all other keys to the new meta property
• wordloom.text_item class now renamed wordloom.language_item
• lang arg on wordloom.language_item (formerly text_item) initializer renamed deflang

Removed

• embedding.pgvector_data_doc (now just embedding.pgvector_data)
• conjunctive option for tags searching now removed, in favor of a query filter mix-in approach

0.8.0

[0.8.0]

Added

• llm_wrapper.llama_cpp_http_chat & llm_wrapper.llama_cpp_http; llama.cpp low-level HTTP API support
• llm_wrapper.llama_response class with flexible handling across API specs
• window init param for for embedding.pgvector.MessageDB, to limit message storage per history key

Changed

• Deprecated first_choice_text & first_choice_message methods in favor of first_choice_text attributes on response objects
• Clarify set quite setup docs

0.7.1

[0.7.1] - 2024-02-22

Added

• MessageDB.get_messages() options: since (for retrieving messages aftter a timestamp) and limit (for limiting the number of messages returned, selecting the most recent)

Changed

• PGVector users now manage their own connection pool by default
• Better modularization of embeddings test cases; using conftest.py more
• pgvector_message.py PG table timstamp column no longer a primary key

Fixed

• Backward threshold check for ogbujipt.embedding.pgvector_data_doc.DataDB