AI as Your Co-Developer

Automating Schemas, Quality Checks, Ingestion

QR to Access Slides

Scan for slides & resources

Prudhvi Krovvidi — Data Scientist, Gramener

30 min • Experiential talk + live demos

Agenda (30 min)

  • Framing — 5m
  • Demo: SchemaForge — 15m
  • Reflections & Q&A — 10m
Agenda Timeline

30-Minute Journey

Framing: Beyond Autocomplete

  • AI now helps design pipelines, tests, and reasoning — not just snippets.
  • Goal: show end-to-end experiments that are deployable and useful.
Beyond Autocomplete

Traditional vs Advanced AI

What you'll take away

  • How AI can output:
    • DBT-ready schemas
    • Test rules
    • Ingestion scripts
    • Many more
  • Patterns & guardrails for production usage.
Key Takeaways

Key Learning Outcomes

End-to-end Information Flow

Information Flow Overview

Demo 1 — SchemaForge

  • Input: raw CSV / sample data
  • Outputs: DBT models, test rules, ER diagram, optional Python ETL
  • Goal: immediately usable artifacts for pipelines
Schema Overview

Repo: github.com/prudhvi1709/schemaforge

SchemaForge in Action

  • Schema inference from CSVs / JSONs → column types, null checks
  • DBT model auto-generation → models + YAML tests
  • ER diagram export → quick visualization of relationships
  • Python ETL pipelines → Pandas + SQLAlchemy ready
SchemaForge Data Ingestion

Data Ingestion Process

SchemaForge Relationships

ER Relationships

Information Flow Overview

End-to-end Information Flow: Data Ingestion to Analytics

SchemaForge — Example Output

DBT Rules SQL

DBT Rules (SQL)

DBT Rules YAML

DBT Rules (YAML)

Reflections & Lessons

  • Trade-offs:
    • Speed vs. Correctness
    • Hallucination vs. Structure
  • Reliability:
    • Guardrails (Tests, Validation, Review Loops)
    • Human-in-the-loop
  • Integration pattern:
    • AI → Scaffold → Validate → Deploy
Trade-offs & Guardrails

AI-Human Collaboration

Key Takeaways

  • AI can produce production-ready artifacts (DBT models, ETL code)
  • Guardrails + human-in-loop make it usable in real workflows
Final Key Takeaways

Core Insights

Try the Repos & Collaborate

  • SchemaForge:
    github.com/prudhvi1709/schemaforge
  • Open to contributions, ideas, and real-world use cases!
GitHub Repositories

Open Source Projects

Q & A

I'd love your questions, thoughts, or feedback!

Let's connect and collaborate on open-source AI tools.

QR Code

Scan to connect

Prudhvi Krovvidi

Prudhvi Krovvidi

Data Scientist, Gramener