Senior Data Engineer

Job Snapshot

Why You’ll Love AgencyBloc

We solve problems for our customers. We give ourselves the autonomy to be awesome. We will be human. We are maniacal about success. This is the culture you’ll be a part of if you join the AgencyBloc Team. We are the #1 Recommended Insurance Industry Growth Platform serving the benefits and senior market space. We help grow our clients’ business with a suite of solutions that can be strategically utilized to support the operational and growth needs of independent insurance agencies, GAs, IMO/FMOs, call centers, and carriers.

Job Description

As a Senior Data Engineer, you will be the senior hands-on builder of AgencyBloc's cloud data platform, implementing the architecture, standards, and patterns defined by the Data Architect and turning them into reliable, production-grade data products.

You will design and build the pipelines, data models, and frameworks that power our Databricks lakehouse on AWS, moving data from our SaaS applications and operational systems (transactional databases, CRM, support, and other internal sources) through bronze, silver, and gold layers to serve BI, analytics, and AI workloads.

This role operates at a senior implementation level: you own delivery of complex data initiatives end-to-end within the established architecture, raise the engineering bar through code review and mentorship of Data Engineers and Data Developers, and contribute practical, build-informed feedback that shapes the platform's standards and roadmap.

Responsibilities:

• Implement data quality validation, reconciliation, and remediation within pipelines according to platform quality standards, ensuring issues are caught before they reach consumers.
• Apply governance standards in day-to-day work, including cataloging, lineage, classification, documentation, and ownership metadata for the datasets you build.
• Implement secure data handling in accordance with security and compliance requirements, including access controls, encryption, masking, and PII handling for sensitive and regulated data (SOC 2 and related standards).
• Apply data retention, archival, and lifecycle rules to the datasets and pipelines you own. 

Observability & Operational Excellence

• Instrument pipelines with monitoring, alerting, and logging aligned to platform observability standards, including SLAs/SLOs for freshness, completeness, and reliability.
• Triage, troubleshoot, and resolve pipeline failures and data incidents, performing root-cause analysis and implementing preventive fixes.
• Build with cost in mind: right-size compute, tune Spark workloads, and monitor the cost profile of the pipelines you own, flagging optimization opportunities.

Platform Standardization & Leadership

• Mentor Data Engineers and Data Developers through code review, pairing, and technical guidance, raising the team's engineering standards.
• Serve as a technical escalation point for complex pipeline, modeling, and performance problems.
• Participate in architecture reviews, providing implementation-level input on feasibility, effort, and operational impact.
• Partner with analytics, product, and engineering stakeholders to understand data needs and translate them into well-modeled, trustworthy datasets.

AI & Analytics Enablement

• Build and maintain the datasets, features, and serving structures that power BI dashboards and AI/ML workloads on the lakehouse.
• Implement patterns to deliver trusted, well-governed data to AI and analytics use cases, as defined in the platform strategy.

Skills/Education/Experience:

• Bachelor's degree in Computer Science or equivalent experience preferred.
• 7+ years of experience in data engineering or analytics engineering, with at least 2 years operating at a senior level (leading delivery of significant data initiatives).
• Strong hands-on production experience with Databricks and/or Spark, including PySpark development, Delta Lake, and workload tuning; Unity Catalog experience is a plus.
• Experience building pipelines on AWS (e.g., S3, IAM, RDS, Glue or comparable services); dual-cloud exposure (Azure) is a plus.
• Experience implementing Medallion (bronze/silver/gold) or comparable layered lakehouse architectures.
• Strong data modeling skills for analytical use cases (dimensional modeling required; exposure to Data Vault or similar history-preserving patterns is a plus).
• Experience extracting and modeling data from relational/OLTP source systems (e.g., MySQL), including CDC or incremental load patterns and handling imperfect source data.
• Expert-level SQL and strong Python, with software engineering fundamentals: version control, testing, code review, and CI/CD for data pipelines (e.g., GitHub Actions).
• Experience implementing data quality checks, pipeline observability, and monitoring/alerting in production.
• Working knowledge of data security practices, including access management, encryption, masking, and PII handling in regulated environments (SOC 2 or similar).
• Experience with orchestration tools (e.g., Databricks Workflows, Airflow, or comparable).
• Strong communication skills, including the ability to explain technical decisions and data issues to non-technical stakeholders.
• Experience working in insurance, InsurTech, or other regulated industries is a plus.
• Experience supporting AI/ML data needs (feature datasets, training data preparation) is a plus.

Applicants for employment in the US must have work authorization that does not now or in the future require sponsorship of a visa for employment authorization in the United States.

---

More on AgencyBloc:

AgencyBloc provides a suite of online software solutions that help insurance agencies increase efficiency, enhance customer satisfaction, control costs, and stay competitive. We currently have customers nationwide. Core values of our company include: 

• Client-centric organization focused on delivering great customer service
• Positive, collaborative team environment with coworkers who genuinely care about their work
• High level of professionalism in everything we do
• Continuous teaching, learning, and growth for all team members as they share in the journey that is the story of our company

Form not loading? Send your resume to resumes@agencybloc.com to apply.

Apply Now