
Data Connectors in Knowledge Base
Create knowledge bases with structured and unstructured data to enhance AI agent’s performance
THE CHALLENGE
I am constantly stitching together pipelines across multiple tools and environments just to make data usable. All the context switching slows down how quickly we can build and iterate.
Senior Data Engineer
Knowledge Technology & Internal Search
Data engineers face major challenges in deploying generative AI applications due to limited data connectivity and fragmented workflows. The inability to directly integrate data from Redshift and S3 delays deployment, while constant context-switching between tools reduces productivity. Incomplete access to organizational data weakens the accuracy of AI agents, and the need to duplicate or transform data for knowledge bases drives up both storage and computational costs.
ROLE
Lead UX designer
DURATION
4 months (February 2025 - June 2025)
KEY RESPONSIBILITIES
Performed parity analysis between Bedrock in AWS Console and SageMaker Unified Studio and documented usability gaps to inform product requirements
Conducted usability tests and refined designs based on feedback
Partnered with PM and Engineering to prioritize MLP requirements and inform future roadmap
TEAM
UX: 2 (1 UXD, 1 UXR)
PM: 1
Engineering: 5 (1 SDM, 2 FEs, 2 BEs)
DESIGN PROCESS
Research
During the research phase, I conducted parity analysis between Bedrock in AWS Console and SageMaker Unified Studio to identify usability gaps and inform improvements for the data connectors feature launch. Across the research and iteration phases, I joined two customer sessions and conducted five usability tests with data engineers to uncover pain points, synthesizing all findings in a centralized document for stakeholders.

Concept generation
Throughout the research and concept generation stages, I created multiple concept variations to get user and customer feedback and iterated based on feedback.

Concept evaluation
For the usability testing sessions, I built an end-to-end prototype that incorporated prior user and customer feedback to validate the full user journey with data engineering users.

Final design
In the final design phase, I refined the solutions based on all session feedback and prepared the deliverables for development handoff.

key design decisions
Data source type selector
I want to get a sense of what data types are supported.
Data Platform Architect, Data Platform & Customer Engineering
The existing Project S3 file selector was limited to S3 imports. To enable future scalability, I designed a Project data sources selector that accommodates additional data sources with expanded real estate to support the varied configurations different source types require.

Data source selector
As a user, I want to know what data types I have knowledge of. If the consumer doesn't know where the data lies, then the system can give information on where to find and add new data.
Senior Data Engineer, Internal Search
The new data source selector groups connectors by category, with built‑in scalability for future types.

Within each connector category, data engineers can add a new connection without leaving the current context.

User control on data table selection
Inside the database, I may have different tables that are not correlated to each other so I want to preview each data table and select the right ones to include in the KB.
Senior Data Engineer, Internal Search
Data engineers gain greater control over which data tables and columns to include in the knowledge base, with full visibility into the metadata. This streamlines productivity by eliminating the need to switch tools or manually open Excel and CSV files.

Streamlined annotation
Column names such as text_1, text_2 don't make sense so if the KB uses these columns, I need to annotate them.
Data Engineer, Robotics
In the current Bedrock in AWS Console experience, users must manually enter table and column names to annotate and add them to the knowledge base. The redesign in SageMaker Unified Studio introduces typeahead so data engineers can quickly select tables or columns without memorizing names, streamlining the annotation process.

Faster experimentation, more accurate AI agent responses
Directly integrating structured data into the knowledge base eliminates manual data preparation and reduces tool-switching, allowing data engineers to move from concept to deployment much faster with fewer engineering steps.
By grounding agents in comprehensive organizational data, responses become more precise and reliable, enabling data engineers to trust outputs and experiment rapidly. With cleaner, connected datasets, teams can iterate quickly, validate improvements faster, and refine agent behavior with greater confidence.

Multiple ingress points
Mapping the ingress points for viewing, creating, and managing knowledge bases allowed the engineering team to uncover technical constraints early, reducing project risks later on.


Data connectors in generative AI flow builder
The same data connector component is extended across other product surfaces to meet evolving customer needs, reducing engineering rework and accelerating time to value.



