Data Infrastructure & Integration

• Design and implement data integrations and ingestion processes for internal and external data sources

• Build and maintain scalable data pipelines for ingesting, processing, and transforming unstructured data sources (customer feedback, documents, multimedia content)

• Develop data models and mapping rules to transform raw data into actionable insights and structured outputs

• Architect and implement semantic layers that integrate analytics data from multiple sources efficiently

AI/ML System Integration

• Develop and maintain robust backend APIs and services supporting the entire prompt-to-answer workflow

• Implement and optimize retrieval logic including vector search, hybrid search, and advanced information retrieval techniques

• Manage document ingestion pipelines including parsing, OCR, chunking, and embedding generation

• Support integration of various LLM providers (OpenAI, Azure AI, Anthropic) with internal business data sources

Infrastructure & Operations

• Ensure reliability, scalability, and low latency of AI response generation systems

• Implement data governance policies and procedures for responsible and ethical use of data in AI applications

• Develop data quality monitoring and validation processes specifically for AI/ML datasets, including bias identification and mitigation

• Build and maintain monitoring, alerting, and observability systems for AI infrastructure

Collaboration & Documentation

• Collaborate with analytics and data science teams to understand requirements and deliver solutions

• Work with data scientists to ensure data is available in appropriate format and quality for model training and deployment

• Maintain comprehensive documentation including data models, mapping rules, and data dictionaries

• Partner with internal business stakeholders, technology resources, and external vendors