Design and maintain scalable data pipelines tailored to LLM requirements, including preprocessing unstructured text data from various sources, implementing chunking strategies, and optimizing embedding generation for vector databases.
Build and manage data infrastructure, including data warehouses, data lakes, and streaming solu...
