Skip to content

Enhanced Ingestion with Named Graph Restrictions + Chat Service #19

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 104 commits into
base: main
Choose a base branch
from
Open

Enhanced Ingestion with Named Graph Restrictions + Chat Service #19

wants to merge 104 commits into from

Conversation

tekrajchhetri
Copy link
Collaborator

@tekrajchhetri tekrajchhetri commented Feb 21, 2025 •
edited
Loading

Enhanced Ingestion with Named Graph Restrictions + Chat Service

Features Implemented

  • JSON-LD Ingestion: Supports ingestion of single or multiple JSON-LD files.
  • Turtle Ingestion: Enables ingestion of single or multiple Turtle files.
  • Restricted Ingestion: Ensures that data is only ingested into pre-registered named graphs, preventing unintended ingestion of arbitrary data.
  • PROV Metadata Tracking:
    • Captures provenance data for each ingestion activity.
    • Records metadata about the named graphs.
    • Uses the PROV ontology and DCTERMS (Dublin Core) for recording metadata and provenance.
    • Implement chat service

@tekrajchhetri
updated shared.py to add new helper function convert_ttl_to_named_graph
096e809
@tekrajchhetri
Updated RabbitMQ listener to handle, i.e., consume, the published mes…
816e1d9 
…sage from the publisher.
@tekrajchhetri
new functions added to attach provenance information about ingestion …
2f109c7 
…activity
@tekrajchhetri
checks added to ensure required parameters are present
fedb7b7
@tekrajchhetri
namespace updated from test example.org to brainkb.org
ad7f206
@tekrajchhetri
update the test connection method to handle empty result for success
3a9413c
@tekrajchhetri
endpoint that handles insertion of data to graphdatabase
08f9ca2
@tekrajchhetri
new endpoint added
bc8d263
@tekrajchhetri
removed /query/insert-jsonld endpoint
2ec9f6f
@tekrajchhetri
updated pydantic schema.
b27e37f
@tekrajchhetri
code update to end-to-end raw json-ld data ingestion.
c645a04
@tekrajchhetri
query service updated to fetch the registered named graphs
ec5fe1f
@tekrajchhetri
shared.py updated to add the metadata about the named graph
d322e35
@tekrajchhetri
NamedGraphSchema added
c66d071
@tekrajchhetri
endpoint renamed and new endpoint to register the named graph added.
047b3f7
@tekrajchhetri
fixed issue - unable to upload multiple json-ld/turtle files
136ea9d
@tekrajchhetri
logger added
f261d7e
@tekrajchhetri
updated configuration.py to read QUERY_SERVICE_BASE_URL for ingestion.
98f033a
@tekrajchhetri
updated pydantic schema for named graphs
d380973
@tekrajchhetri
new function added to check if the targeted named graphs exists, i.e.…
331300d 
…, registered.
@tekrajchhetri
code updated restrict ingestion of ttl/jsonld (raw+file) to registere…
6edb19d 
…d graph.
@tekrajchhetri
added caching mechanism to store user chat.
94c7355
@tekrajchhetri
Update pydantic_models.py
248c8bc
@tekrajchhetri
bican anatomical model for query generation purpose
290d41f
@tekrajchhetri
updated requirements.txt
d86c885
@tekrajchhetri
Integrated LLM. @todo: Query generation based on schema + add predefi…
db19fcf 
…ned queries.
@tekrajchhetri
updated configuration to make it robust + added new configurations ne…
4c89353 
…cessary for chat service
@tekrajchhetri
improved logging features
fe1996a
@tekrajchhetri
improved database setting
faaf370
@tekrajchhetri
get_postgres_cache_settings to configuration
5e6af45
@tekrajchhetri
updated security to reflect the updated configuration
55881ee
@tekrajchhetri
added features to support template based query for chat
c90ee3f
@tekrajchhetri
template based query features added
f111026
@tekrajchhetri
cache init
6daccf8
@tekrajchhetri
Update requirements.txt
9510ecb
@tekrajchhetri
schema guided prompt generation
17692e6
@tekrajchhetri
genome annotation
9b1c278
@tekrajchhetri
Update requirements.txt
f48100b
@tekrajchhetri
default query added + llm based automated query fixing mechanism added
804ef44
@tekrajchhetri
Intelligent query understanding and parameter extraction • Automatic…
29c893c 
… error fixing for SPARQL queries • Streaming responses for real-time interaction • Context-aware follow-up question handlin
@tekrajchhetri
docker deployment bash script
1e91331
@tekrajchhetri
JWT added
95dd28e
@tekrajchhetri
sandbox url added
75dcdd8
@tekrajchhetri tekrajchhetri changed the title Enhanced Ingestion with Named Graph Restrictions Enhanced Ingestion with Named Graph Restrictions + Chat Service Jul 29, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@aaronkanzer aaronkanzer aaronkanzer left review comments

@puja-trivedi puja-trivedi Awaiting requested review from puja-trivedi

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@tekrajchhetri @aaronkanzer