Sr. Director of Data Engineering

GENERAL FUNCTION

As the Sr Director of Data engineering, you will lead the team responsible for all the data engineering for the enterprise. This role will work closely with business partners and other engineering teams to ensure that all data, metrics, and operational performance data is accurate, timely, and accessible for analysis, reporting, and operational use.

DUTIES/RESPONSIBILITIES

• Build and lead the onshore & offshore data engineering team.
• Collaborate with business and technical partners across the enterprise to insource data, create data products, and make them available for use
• Refine and plan data engineering projects to support partner’s needs
• Develop and own the data processing framework & delivery of our securitization process
• Develop and own the execution of the single source of truth framework
• Ensure pipelines are properly documented and team SOPs are followed
• Partner with technology to maintain optimal data history in the transactional databases
• Establish controls and monitoring to ensure timely quality and delivery of data products
• Mentor, manage, and motivate the data engineering team
• Evaluate and engage with external vendors for new tooling or staff augmentation
• Complete the data lake migration, pipelines, and tooling to Snowflake
• Collaborate on the establishment of enterprise data governance

EDUCATION

• Bachelor’s degree in Computer Science, Data Engineering, Information Systems, or related field (Master’s preferred).

EXPERIENCE

• 15+ years of experience in data strategy, architecture, governance, & engineering
• 5+ years developing and managing teams
• 7+ years with hands on experience in distributed data ecosystems such as Apache Spark (DataBricks), Hadoop or Snowflake. Understand cluster optimization techniques and appropriate strategy.
• Demonstrated data modelling experience for operational efficiency on lake houses such as Kimball Dimensional modelling, Vault etc.
• Expose to setting up Data Governance blueprints to solve for Compliance, Data Lifecycle management, asset classification, exposure to data masking techniques, security posture setup on ecosystem, implement data stewardship.
• Demonstrated work experience in Analytics environments and BI tools such as Jupyter (With all relevant python libraries), Tableau or Sigma
• Demonstrated work experience with Snowflake is plus but not mandatory.
• 10+ years of experience in programming languages like Python or Java or .Net. SQL is a must
• Strong understanding on Cloud footprint to leverage services for data integration - Function Apps, Lambda, Step Function etc.
• Demonstrated work experience with one of the Orchestrator tools such as ADF, AWS Glue, Apache Airflow.
• Experience leveraging streaming platforms such as Apache Kafka/ Nifi, Azure Event hubs or equivalent
• Scaled up CI/ CD for data environment - pipeline artifacts and code migration
• Experience is leveraging API Protocols for data extraction and ingestion into lake houses.
• 7+ years of experience in building engineering frameworks for data processing, Data quality. Have a strong idea on Meta data driven ELT frameworks.

SKILLS

• Ability to manage large initiatives
• Functional knowledge of financial instruments such as Installment Loans, Payday Loans and data domains such as loans, underwriting, collection, promotions, call center data, Fraud, etc.
• Demonstrated awareness of using the right security protocols for data sharing - Encryptions, authentication & authorization
• Expert knowledge of Big Data technologies such as Data Bricks, Event driven data processing using Function Apps/Lambda is preferred
• Understanding of Web services (SOAP, XML, UDDI, WSDL)
• Excellent problem-solving and analytical skills.
• Ability to manage and develop a data engineering team.
• Ability to collaborate and work across teams.
• Strong communication skills for both technical and non-technical audiences