Senior Database Engineer (Vector Database)

• Database

• Remote

• Contract

About the job:

Title – Senior Database Engineer (Vector Database)
Start date: Immediate
Position Type: Contract
Location: Remote across Canada/USA
  
About the Role: 
We are seeking a Senior Database Engineer with expertise in building and managing on-premise vector databases and traditional relational databases. This role focuses on securely managing and storing codebases in databases and implementing robust vector database systems to support AI/ML workflows. 
As a core team member, you will architect scalable and efficient database solutions, ensuring all code and metadata are securely managed while optimizing vector databases for high-performance similarity searches and AI model requirements. 
  
Key Responsibilities: 
  
Database Design and Implementation: 

• Architect and implement on-premise vector databases (e.g., Milvus, Pinecone, Weaviate, or custom solutions) for similarity search and AI/ML model integration. 
• Design and manage traditional relational databases for code storage, metadata management, and other structured data. 

  
Code Management in Databases: 

• Develop efficient schema designs and workflows to securely store and manage large-scale codebases within databases. 
• Enable seamless querying, versioning, and retrieval of code artifacts for CI/CD pipelines. 

  
Database Optimization: 

• Optimize database performance for high-availability, low-latency operations, and scalability. 
• Implement indexing techniques such as HNSW or IVF for efficient vector searches. 

  
Infrastructure Management: 

• Deploy and manage databases on-premise, ensuring security, reliability, and resource optimization. 
• Collaborate with infrastructure teams to integrate vector and relational databases into on-premise or hybrid environments. 

  
Integration with AI/ML Systems: 

• Collaborate with AI/ML teams to integrate vector databases for embedding storage and similarity searches. 
• Build data pipelines to populate and update vector embeddings in real time or batch processes. 

  
Security and Compliance: 

• Ensure all database solutions comply with organizational security policies and standards. 
• Implement encryption, secure access, and backup strategies for sensitive data. 

  
Monitoring and Maintenance: 

• Set up monitoring tools and processes to track database health, performance, and anomalies. 
• Handle upgrades, migrations, and troubleshooting with minimal downtime. 

  
Key Requirements: 
  
Experience: 

• 7+ years of experience in database engineering or related roles. 
• At least 3 years of experience working with vector databases and AI/ML data management. 

  
Technical Skills: 
  
Database Expertise: 

• Hands-on experience with vector databases such as Milvus, Weaviate, Pinecone, or FAISS. 
• Strong expertise in relational databases (e.g., PostgreSQL, MySQL, or MS SQL) and NoSQL databases. 

  
Vector Database Techniques: 

• Proficiency in implementing indexing algorithms such as HNSW, IVF, or PQ for vector similarity searches. 
• Familiarity with embedding generation and integration into vector databases. 

  
Infrastructure Management: 

• Experience with deploying and managing on-premise database systems. 
• Proficiency in containerization and orchestration tools (e.g., Docker, Kubernetes). 

  
Programming Skills: 

• Strong skills in SQL and database scripting languages. 
• Proficiency in Python for database management and integration tasks.

  
Security and Optimization: 

• Deep understanding of database security practices, including encryption, access control, and auditing. 
• Experience in performance tuning for large-scale databases. 

  
Preferred Qualifications: 

• Knowledge of embedding generation using models like OpenAI, Sentence Transformers, or similar. 
• Experience integrating vector databases with LLM-based systems for retrieval-augmented generation (RAG). 
• Familiarity with data pipelines and ETL processes for embedding updates. 

  
Soft Skills: 

• Strong problem-solving and troubleshooting abilities. 
• Excellent collaboration and communication skills to work with cross-functional teams. 
• Leadership experience in managing projects or mentoring junior team members.