Data Engineer

Data Engineer with Bachelor’s Degree in Computer Science, Computer Information Systems, Information Technology, or a combination of education and experience equating to the U.S. equivalent of a Bachelor’s degree in one of the aforementioned subjects.

Job Duties and Responsibilities:

• Design and develop Data lake and exploration with security policies, custom integration with IAM Systems (like Okta AD, PingID).
• Manage relationships with key technology, business partners and product owners and other stake holders.
• Build integration with Single Page Apps, Mobile Apps, Third Party Systems using OAuth2, SSO.
• Build API, Common reusable component and automation framework for Client’s intranet access to physical services running in sandbox or test environment.
• Implementation of Data Ingestion and Analytics and digitization suit on real time data analytics projects, analyze and configure technical AWS cloud Databricks platform integration requirements and bring deep knowledge and experience in designing for and implementing solutions in the Cloud (AWS).
• Implementation of Serverless Technologies like Lambda and CICD using AWS CodePipeline, CodeCommit, CodeBuild. CodeDeploy and adopt full adoption of modern software engineering and delivery practices with design principles and patterns, Agile, Big data Spark using Scala and Databricks, Stateless Design Java REST/ Spring Boot based Microservices, Containerization etc.
• Ensure that CI/CD pipeline covers lifecycle management needs for enterprise use cases (like covering APIs deployed in the legacy API Management environment).
• Analyze and perfo1m complex solution feasibility studies of the above-mentioned projects, process technical design reviews, serve as Big data SME and support respective managers from technology point of view for all above said projects.
• Analyze and design Data Lake and computation Layer for Analytics and machine learning team, reporting teams, technical validations, Substitutions required for all the above Big data Enterprise technical renovation projects.
• Explore and design Big data latest cutting-edge technologies like AI, strategy, and management, including data architecture, governance, quality, and technologies

Expertise/ Skills required for the position:

• 5+ years of experience in building software enterprise software applications through Java J2EE based Technology Stack and Big data Ingestion framework and distribution framework using Hadoop, spark and scala.
• 4+ years’ experience working with Spark using Scala and Java on Mapr Distribution on Mapr FS, Hortonworks, and Big data tools including Hive, Drill, Hue, Kafka, Solace with HBase, Mongo DB Mapr-DB(Binary & JSon) No SQL databases.
• Having very strong knowledge in working with Object Oriented programming languages such as Java, Microsoft, Spring MVC, Spring Boot Microservices application.
• Strong experience in Databricks and Delta tables, relational database concepts, SQL, and procedural languages; object-oriented design; Enterprise, distributed computing and WEB-based computing methods; and design patterns.
• Strong experience in Data Lake, Delta Lake, Data computation and Analytics and the concepts of REST services as well as both XML and JSON message formats.
• Bring deep knowledge and experience in designing for and implementing solutions in the Cloud (AWS).
• Proficient in Continuous Integration (CI) and Continuous Deployment (CD) pipelines using Jenkins CI.
• Strong analytical and problem-solving skills as well as the ability to decompose complex problems and perform root cause analyses.
• Work in a collaborative environment.
• Experience with various testing methodologies and strategies: Test Driven Development (TDD) implemented with JUnit, Mock objects, Stubs, Test suites, Test harness.
• Experience working with agile team tools (GitHub, JIRA, Scrum).
• Experience working with Eclipse IDE and IntelliJ IDE with Maven and Gradle build tools.
• Ability to self-organize, prioritize and handle multiple priorities without compromising on quality.
• Expert analytical skills of complex solution feasibility studies of the above-mentioned projects, process technical design reviews, serve as big data Technical consultant and support respective managers from technology point of view.
• Expert knowledge of Data Lake and Analytics with AWS cloud Platform and Management Controlling teams with real time data handling training, support with quality data for Analytics, Machine learning and AI team, guiding through new BI and reporting processes for timely performance and making better decisions for the Customer needs and market preferences.

Technologies Involved:

Big Data, AWS Cloud, Spark, Databricks Hadoop – Spark 2.2.4, Scala 2.12, Hive, Drill, Airhflow, Redhat Ceph, Open IO,Scality Ring, Kafka, Solace, Hue, Zookeeper, Core Java, J2EE, Spring MVC, Spring Boot, Spring Hibernate, Spring (JPA, Rest) Microservices Swagger API, Cloud Computing – AWS, S3 compatible API’s, Knowledge on Kubernetes and containerization, SQL Server, Oracle 10g & 11g, DB2, Teradata, No SQL DB – Hbase, Mapr DB, Mongo DB.

Work location is Portland, ME with required travel to client locations throughout USA.

Rite Pros is an equal opportunity employer (EOE).

Please Mail Resumes to:

Rite Pros, Inc.

565 Congress St, Suite # 305

Portland, ME 04101.

Email: [email protected]