Senior Software Engineer (Collections)

The Collections team provides around-the-clock insights from conversations within threat actor channels and illicit online communities. These communications provide insight into a broad spectrum of activity, threat actor tactics, techniques, and procedures (TTPs). Accessing collections data through the Flashpoint Intelligence Platform provides additional insights for security teams to discover and respond to threats in a timely manner, thereby reducing risk to their organizations.

Trusted by governments, commercial enterprises, and educational institutions worldwide, Flashpoint helps organizations protect their most critical assets, infrastructure, and stakeholders from security risks such as cyber threats, ransomware, fraud, physical threats, and more. Leading security practitioners—including physical and corporate security, cyber threat intelligence (CTI), vulnerability management, and vendor risk management teams—rely on the Flashpoint Intelligence Platform, comprising open source (OSINT) and closed intelligence, to proactively identify and mitigate risk and stay ahead of the evolving threat landscape. Learn more at www.flashpoint.io. 

Our best engineers are collaborative and selfless. They enjoy writing code but they prioritize mentoring and supporting others. They leave code cleaner than when they found it but don’t insist on particular patterns. They optimize for simplicity and readability. They are involved in the full lifecycle of software development: understanding the requirements, designing the solution, writing the code, testing the code, testing the integration, deploying it to production, and monitoring the deployment. They are fully engaged in the problem domain.

We have a role for you if:

• You are proficient in Python
• You are proficient in CI/CD
• You enjoy for working in a close-knit, highly communicative and collaborative team
• You have a coding personality that excels at debugging and applying glue in engineering gaps
• You are comfortable with non-deterministic input
• You have a humble attitude and persistence to learn
• You have the ability to write documentation, task tickets, and requests for comments
• You have experience with mentoring junior developers, product development, and the individual contributor/manager pendulum

What you will get to do on our team:

• Develop, own, and maintain scraping and parsing tools such as Python Requests, BeautifulSoup, and headless browsers
• Develop, own, and maintain streaming pipelines using tools like Kafka, AWS MSK, and Google PubSub
• Be hands on with cloud technologies from providers like AWS and GCP
• Develop hands on experience with container technologies like Docker and Kubernetes
• Python testing frameworks like pytest
• Search databases like Elasticsearch and Solr
• Key/Value stores like Redis, Cassandra, and BigTable
• Monitor with tooling like ElasticAPM, Prometheus, and Grafana
• Increase proficiency with data warehousing tools such as BigQuery
• Master business Intelligence platforms like Tableau, Looker, and Snowflake

What you will achieve:

• Within 30 days
• Configure your environment and familiarize yourself with the services and processes the team uses via onboarding sessions
• Make basic source additions to the collections system
• Within 60 days
• Begin working on infrastructure projects and maintenance projects to further deepen your understanding of the department’s role within the organization
• Work on more challenging source additions that require non-standard solutions
• Expand source addition and maintenance knowledge to more complex source types
• Within 90 days
• Begin leading larger projects with broader impact, potentially directing other engineers on the team within it
• Work on challenging source additions that require non-standard solutions
• Continue to support development teams and improve internal processes/services
• Start on-call shift (weekly shifts approximately every 6-7 weeks)

To be successful in this role:

• Build scraping capabilities to collect data at scale from Dark Web communities
• Drive down total time taken for processing (code optimization and refactoring), deployment (smoother CI/CD pipelines), and other SLAs (delivery to clients via our infrastructure)
• Improve our tooling for monitoring the health and efficiency of our systems
• Write empathetic documentation and run-books to enable your team to be force multipliers for each other
• Help bring in new technologies and develop innovative approaches to the data challenges w