AI Data Engineer(AI數據工程師)

1.Data Analyst： ■Utilizing Large Language Models (LLMs) for data processing, including data cleaning, field alignment, classification, and data augmentation. ■Building Retrieval-Augmented Generation (RAG) systems for database processing, integrating vector databases such as FAISS, Qdrant, and Weaviate. ■Implementing vector, semantic, and multimodal search technologies to enhance intelligent information retrieval and data understanding.

2.Design, Create, and Maintain Kubernetes Clusters for AI Applications： ■Manage container images across multiple GCP environments. ■Ensure efficient deployment and scaling of AI applications,using K8S to deploy container service for AI Application.

3.Manage Cloud Resources： ■Oversee resources across GCP, AWS, and Azure Open AI. ■Manage Identity and Access Management (IAM) policies and roles. ■Implement and monitor cloud security measures. ■Optimize cloud resource usage and cost. ■Automate infrastructure provisioning and management using tools like Terraform and CloudFormation.

4.Integrate and Host Notification & Queue Services： ■Utilize services such as Amazon SQS, Google Cloud Pub/Sub, and RabbitMQ.

5.Design and Manage Vector Databases： ■Work with databases like Milvus,AI Vector DB and Pinecone for LLM vector search and query.

6.APM and Tracing Systems Integration and Management： ■Implement and manage Application Performance Monitoring (APM) systems, such as Datadog、Databrick、 and Prometheus, for monitoring a.pplication performance. ■Set up tracing systems like Jaeger and Zipkin to monitor usage and uptime.