AI Data Engineer(AI數據工程師)
1.Data Analyst: ■Utilizing Large Language Models (LLMs) for data processing, including data cleaning, field alignment, classification, and data augmentation. ■Building Retrieval-Augmented Generation (RAG) systems for database processing, integrating vector databases such as FAISS, Qdrant, and Weaviate. ■Implementing vector, semantic, and multimodal search technologies to enhance intelligent information retrieval and data understanding.
2.Design, Create, and Maintain Kubernetes Clusters for AI Applications: ■Manage container images across multiple GCP environments. ■Ensure efficient deployment and scaling of AI applications,using K8S to deploy container service for AI Application.
3.Manage Cloud Resources: ■Oversee resources across GCP, AWS, and Azure Open AI. ■Manage Identity and Access Management (IAM) policies and roles. ■Implement and monitor cloud security measures. ■Optimize cloud resource usage and cost. ■Automate infrastructure provisioning and management using tools like Terraform and CloudFormation.
4.Integrate and Host Notification & Queue Services: ■Utilize services such as Amazon SQS, Google Cloud Pub/Sub, and RabbitMQ.
5.Design and Manage Vector Databases: ■Work with databases like Milvus,AI Vector DB and Pinecone for LLM vector search and query.
6.APM and Tracing Systems Integration and Management: ■Implement and manage Application Performance Monitoring (APM) systems, such as Datadog、Databrick、 and Prometheus, for monitoring a.pplication performance. ■Set up tracing systems like Jaeger and Zipkin to monitor usage and uptime.