Job Summary: We are looking for an experienced Data Engineer to join our team. In this role, you will design and implement high-performance data ingestion pipelines from multiple sources, leveraging Apache Spark and/or Azure Databricks. You will also be responsible for delivering and presenting proofs of concept to stakeholders and developing scalable, reusable frameworks for ingesting geospatial data sets. This is an exciting opportunity to contribute to cutting-edge data engineering initiatives in a fast-paced environment. Key Responsibilities: Design and Implement Data Pipelines: Build and optimize high-performance data ingestion pipelines from diverse sources using Apache Spark and/or Azure Databricks, ensuring scalability and reliability. Proofs of Concept: Deliver and present proofs of concept for key technology components, showcasing new approaches and solutions to project stakeholders. Geospatial Data Integration: Develop scalable and reusable frameworks for efficiently ingesting and processing geospatial data sets, supporting both structured and unstructured data formats. Collaborate with Teams: Work closely with cross-functional teams (data scientists, analysts, product owners) to gather requirements and ensure successful integration of data pipelines. Optimize Performance: Continuously monitor and improve the performance, scalability, and cost-efficiency of the data pipelines and ingestion frameworks. Ensure Data Quality: Apply best practices to ensure data quality, security, and integrity across the entire ingestion process. Required Qualifications: Experience: Proven experience in data engineering, specifically in building data pipelines with Apache Spark and/or Azure Databricks. Geospatial Data Expertise: Familiarity with handling geospatial data sets, including their structure and integration techniques. Data Engineering Frameworks: Experience in developing scalable and reusable data frameworks for complex data processing. Programming: Proficiency in Python, Scala, or Java for building data pipelines and processing large data sets. Cloud Technologies: Experience with cloud platforms, especially Azure, and familiarity with data processing tools such as Azure Data Lake, Databricks, or HDInsight. Collaborative Skills: Strong communication skills and experience working in cross-functional teams. Preferred Qualifications: Experience with Apache Hadoop, Kafka, or other big data frameworks. Familiarity with machine learning pipelines or data processing in AI/ML environments. Experience with ETL/ELT processes and data warehousing solutions Certifications (if applicable): Microsoft Certified: Azure Data Engineer or other relevant certifications in data engineering or cloud technologies. Education: Bachelors Degree JobRialto
Requires a Bachelors or Technology degree in Engineering or a related specialized area/field, or equivalent (4 years job-related experience). Must have d istinct knowledge of the structure and content of the English language including the meaning and spelling of words,...
...campaigns and allow them to take care of existing paid media clients and also grow as they bring in Digital Neighbor is on the hunt for a PPC Strategist with a special combination of intermediate digital marketing strategy, Search Ads, Social Ads, and account development....
...rapidly growing throughout the United States! They are seeking an Accounting Manager who is looking to grow within an organization and who... ...projects and initiatives as needed. Qualifications: ~ Bachelors degree in Accounting or Finance. ~ CPA or MBA preferred....
...ServiceNow HRSD module * Experience interfacing with Human Resources Systems of Record (i.e., SuccessFactors, Workday, UltiPro, PeopleSoft, SAP, etc.) * Experience with SDLC processes implementing in a variety of project methodologies: waterfall, iterative,...
...Welders and Fabricators(Modesto, Ca) compensation: Based on Experience employment type: full-time job title: Misc. Steel Fabricators... ...work overtime, with occasional weekends as needed. - Able to travel to jobsites for field work. - Organized and able to multi-...