Screaming in the Cloud

Let’s chat about the Cloud and everything in between. The people in this world are pretty comfortable with not running physical servers on their own, but trusting someone else to run them. Yet, people suffer from the psychological barrier of thinking they need to build, design, and run their own monitoring system. Fortunately, more companies are turning to Datadog. Today, we’re talking to Ilan Rabinovitch, Datadog’s vice president of product and community. He spends his days diving into container monitoring metrics, collaborating with Datadog’s open source community, and evangelizing observability best practices. Previously, Ilan led infrastructure and reliability engineering teams at various organizations, including Ooyala and Edmunds.com. He’s active in the open source and DevOps communities, where he is a co-organizer of events, such as SCALE and Texas Linux Fest. Some of the highlights of the show include: Datadog is well-known, especially because it is a frequent sponsor More organizations know their core competency is not monitoring or managing servers Monitoring/metrics is a big data problem; Datadog takes monitoring off your plate Alternate ways, other than using Nagios, to monitor instances and regenerate configurations Datadog is first to identify patterns when there is a widespread underlying infrastructure issue Trends of moving from on-premise to Cloud; serverless is on the horizon How trends affect evolution of Datadog; adjusting tools to monitor customers’ environments Datadog’s scope is enormous; the company tries to present relevant information as the scale of what it’s watching continues to grow Datadog’s pricing is straightforward and simple to understand; how much Cloud providers charge to use Datadog is less clear Single Pane of Glass: Too much data to gather in small areas (dashboards)   Why didn’t monitoring catch this? Alerts need to be actionable and relevant How to use Datadog’s workflow for setting alerts and work metrics Datadog’s first Dash user conference will be held in July in New York; addresses how to solve real business problems, how to scale/speed up your organization Links: Ilan Rabinovitch on Twitter Datadog Docker Adoption Survey Results  

Do you need data captured that let you know when things don’t look quite right? Need to identify issues before they become major problems for your organization? Turn to Threat Stack, which has Cloud issues of its own, and helps its customers with their Cloud issues. Today, I’m talking to Pete Cheslock, who runs technical operations at Threat Stack, which handles security monitoring, alerting, and remediation. The company uses Amazon Web Services (AWS), but its customer base can run anywhere.   Some of the highlights of the show include: Challenges Threat Stack experienced with AWS and how it dealt with them Threat Stack helps companies improve their security posture in AWS Security shouldn’t be an issue, if providers do their job; shared responsibility Education is needed about what matters regarding security, avoiding mistakes Cloud is still so new; not many people have abroad experience managing it Scanning customer accounts against best practices to identify risks Threat Stack’s scanning tool is worthwhile, but most tools lack judgement and perspective Threat Stack offers context between host- and Cloud-based events; tying data together is the secret sauce You shouldn’t have to pay a bunch of money to have a robust security system Good operations is good security; update, patch, track, and perform other tasks Lack of validation about what services are going to be a successful or not Vendor Lock-in: Understand your choices when building your system Pervasiveness and challenge of containerization and Kubernetes Cloud reduces cycle time and effort to bring a product to market Amazon is a game changer with what it allows you to do and solve problems Links: Pete Cheslock Digital Ocean Threat Stack AWS re:Invent Kubernetes

Do you need data captured that let you know when things don’t look quite right? Need to identify issues before they become major problems for your organization? Turn to Threat Stack, which has Cloud issues of its own, and helps its customers with their Cloud issues. Today, I’m talking to Pete Cheslock, who runs technical operations at Threat Stack, which handles security monitoring, alerting, and remediation. The company uses Amazon Web Services (AWS), but its customer base can run anywhere.   Some of the highlights of the show include: Challenges Threat Stack experienced with AWS and how it dealt with them Threat Stack helps companies improve their security posture in AWS Security shouldn’t be an issue, if providers do their job; shared responsibility Education is needed about what matters regarding security, avoiding mistakes Cloud is still so new; not many people have abroad experience managing it Scanning customer accounts against best practices to identify risks Threat Stack’s scanning tool is worthwhile, but most tools lack judgement and perspective Threat Stack offers context between host- and Cloud-based events; tying data together is the secret sauce You shouldn’t have to pay a bunch of money to have a robust security system Good operations is good security; update, patch, track, and perform other tasks Lack of validation about what services are going to be a successful or not Vendor Lock-in: Understand your choices when building your system Pervasiveness and challenge of containerization and Kubernetes Cloud reduces cycle time and effort to bring a product to market Amazon is a game changer with what it allows you to do and solve problems Links: Pete Cheslock Digital Ocean Threat Stack AWS re:Invent Kubernetes

Aurora, from Amazon Web Services (AWS), is a MySQL-compatible service for complex database structures. It offers capabilities and opportunities. But with Aurora, you’re putting a lot of trust in AWS to “just work” in ways not traditional to relational database services (RDS). David Torgerson, Principal DevOps Engineer at Lucidchart, is a mystery wrapped in an enigma and virtually impossible to Google. He shares Lucidchart’s experience with migrating away from a traditional RDS to Aurora to free up developer time. Some of the highlights of the show include: Trade off of making someone else partially responsible for keeping your site up Lucidchart’s overall database costs decreased 25% after switching to Aurora Aurora unknowns: What is an I/Op in Aurora? When you write one piece of data, does it count as six I/Ops? Multi-master Aurora is coming for failover time and disaster recovery purposes Aurora drawbacks: No dedicated DevOps, increased failover time, and misleading performance speed Providers offer ways to simplify your business processes, but not ways to get out of using their products due to vendor and platform lock-in Lucidchart is skeptical about Aurora Serverless; will use or not depending on performance Links: Corey's architecture diagram on AWS Lucidchart Lucidchart’s Data Migration to Amazon Aurora Preview of Amazon Aurora Multi-master Sign Up This is My Architecture re:Invent Digital Ocean

Aurora, from Amazon Web Services (AWS), is a MySQL-compatible service for complex database structures. It offers capabilities and opportunities. But with Aurora, you’re putting a lot of trust in AWS to “just work” in ways not traditional to relational database services (RDS). David Torgerson, Principal DevOps Engineer at Lucidchart, is a mystery wrapped in an enigma and virtually impossible to Google. He shares Lucidchart’s experience with migrating away from a traditional RDS to Aurora to free up developer time. Some of the highlights of the show include: Trade off of making someone else partially responsible for keeping your site up Lucidchart’s overall database costs decreased 25% after switching to Aurora Aurora unknowns: What is an I/Op in Aurora? When you write one piece of data, does it count as six I/Ops? Multi-master Aurora is coming for failover time and disaster recovery purposes Aurora drawbacks: No dedicated DevOps, increased failover time, and misleading performance speed Providers offer ways to simplify your business processes, but not ways to get out of using their products due to vendor and platform lock-in Lucidchart is skeptical about Aurora Serverless; will use or not depending on performance Links: Corey's architecture diagram on AWS Lucidchart Lucidchart’s Data Migration to Amazon Aurora Preview of Amazon Aurora Multi-master Sign Up This is My Architecture re:Invent Digital Ocean

Does your job challenge and motivate you? Does it utilize your skills? Or, are you ready to go job hunting? Do you want an awesome job that is a resume booster? Companies should be supportive of their employees finding a job that matches their skills and interests. Also, when hiring, companies should offer thoughtful processes for interviews.   Today, I’m talking to Sarah Withee, a polyglot software engineer, mentor, teacher, and robot tinkerer. Sarah went job hunting, and after several job interviews, she finally found a job that made her super happy at Arcadia Healthcare Solutions. Sarah compares the interview processes she experienced at big name tech companies that offer Cloud services. Some of the highlights of the show include: Companies sometimes lose sight that even interview interactions need to be a two-way sale Interviews often involve talking to many people; and if several are bad, that forms a negative impression of the company Companies need to provide interview training and follow the same standards Don’t farm out challenging or unfamiliar issues when interviewing candidates Sarah is very competent, but she is new to Cloud platforms; she is like a sponge, who enjoys learning and having a bare knowledge of new technology How HIPAA regulations impact Sarah’s learning and software engineering work; she has to be more aware of security and safety of healthcare data Being a teacher and mentor affects how Sarah learns new things; everybody learns slightly differently In the Cloud space, know which direction you want to go and start with simpler things to learn the basics; focus on what is relevant to what you are working on Links: Sarah Withee on Twitter #speakerconfessions Sarah Withee on Twitter Sarah Withee Blog Sarah Withee Resume Digital Ocean AWS Azure

Does your job challenge and motivate you? Does it utilize your skills? Or, are you ready to go job hunting? Do you want an awesome job that is a resume booster? Companies should be supportive of their employees finding a job that matches their skills and interests. Also, when hiring, companies should offer thoughtful processes for interviews.   Today, I’m talking to Sarah Withee, a polyglot software engineer, mentor, teacher, and robot tinkerer. Sarah went job hunting, and after several job interviews, she finally found a job that made her super happy at Arcadia Healthcare Solutions. Sarah compares the interview processes she experienced at big name tech companies that offer Cloud services. Some of the highlights of the show include: Companies sometimes lose sight that even interview interactions need to be a two-way sale Interviews often involve talking to many people; and if several are bad, that forms a negative impression of the company Companies need to provide interview training and follow the same standards Don’t farm out challenging or unfamiliar issues when interviewing candidates Sarah is very competent, but she is new to Cloud platforms; she is like a sponge, who enjoys learning and having a bare knowledge of new technology How HIPAA regulations impact Sarah’s learning and software engineering work; she has to be more aware of security and safety of healthcare data Being a teacher and mentor affects how Sarah learns new things; everybody learns slightly differently In the Cloud space, know which direction you want to go and start with simpler things to learn the basics; focus on what is relevant to what you are working on Links: Sarah Withee on Twitter #speakerconfessions Sarah Withee on Twitter Sarah Withee Blog Sarah Withee Resume Digital Ocean AWS Azure

Docker went from being a small startup to an enterprise company that changed the way people think about their infrastructure to now, where its relevance is somewhat minimal. The conversation is no longer around the container level. Docker has become commonplace. Today, we’re talking to Jérôme Petazzoni, formerly of Docker. While he was with the company for about 8 years, Docker definitely experienced a roller coaster ride.   Some of the highlights of the show include: Amount of work conducted on the enterprise vs. community editions Docker was so widely adopted because its core technology was open source Challenge is to build a viable business and revenue model for the long run Similarities between Docker and Red Hat open source platforms Docker went from six people working in a garage to having a few hundred employees and $1.3 billion valuation Changes happened, but they were gradual; the changes were necessary to be a profitable and sustainable company Contingent of internal and external people believed that Docker was the answer for whatever problem surfaced; Docker would save you, but not always Balancing Act: Pushing forward with a correct message and regulating enthusiasm Networking and Docker for dummies; confusion and problems of things not working as expected have been resolved Things will continue to shift; Kubernetes and the orchestration battle What was unthinkable, could happen by companies pushing the envelope and making progress Will who you have as your Cloud provider stop mattering? It depends. All major Cloud providers plan to offer managed Kubernetes services and what Jérôme thinks of them Jérôme’s opinion on whether Kubernetes will follow this same path as Docker What does the road ahead look like for infrastructure automation? There is potential and lots of best practices in Cloud environments. Links: Jérôme Petazzoni on Twitter https://jpetazzo.github.io/ Docker Crunch Base Digital Ocean Red Hat

Docker went from being a small startup to an enterprise company that changed the way people think about their infrastructure to now, where its relevance is somewhat minimal. The conversation is no longer around the container level. Docker has become commonplace. Today, we’re talking to Jérôme Petazzoni, formerly of Docker. While he was with the company for about 8 years, Docker definitely experienced a roller coaster ride.   Some of the highlights of the show include: Amount of work conducted on the enterprise vs. community editions Docker was so widely adopted because its core technology was open source Challenge is to build a viable business and revenue model for the long run Similarities between Docker and Red Hat open source platforms Docker went from six people working in a garage to having a few hundred employees and $1.3 billion valuation Changes happened, but they were gradual; the changes were necessary to be a profitable and sustainable company Contingent of internal and external people believed that Docker was the answer for whatever problem surfaced; Docker would save you, but not always Balancing Act: Pushing forward with a correct message and regulating enthusiasm Networking and Docker for dummies; confusion and problems of things not working as expected have been resolved Things will continue to shift; Kubernetes and the orchestration battle What was unthinkable, could happen by companies pushing the envelope and making progress Will who you have as your Cloud provider stop mattering? It depends. All major Cloud providers plan to offer managed Kubernetes services and what Jérôme thinks of them Jérôme’s opinion on whether Kubernetes will follow this same path as Docker What does the road ahead look like for infrastructure automation? There is potential and lots of best practices in Cloud environments. Links: Jérôme Petazzoni on Twitter https://jpetazzo.github.io/ Docker Crunch Base Digital Ocean Red Hat Corey's Heresy in the church of docker talk Kubernetes ZooKeeper Azure

Like migrating caribou, you tend to follow the trends of what clients are doing, which dictates what you work on as a consultant. Today, we’re talking to Lynn Langit, an independent Cloud architect. She is an AWS Community Hero, Google Cloud developer expert, and former Microsoft MVP. Lynn is a lifelong learner, and she has worked broad and deep across all three large providers. These days, she works mostly with Google Cloud and AWS, rather than Azure, because that’s what her clients are using. Some of the highlights of the show include: Differences between the West Coast and global use of Cloud Education is key; Lynn is th co-founder of Teachingkidsprogramming.org Lynn helped create curriculum and resources for school-age children; even her young daughter taught classes on how to code Training for teachers was also needed, so TKP Labs was formed to offer fee-based teacher and developer training Lynn started with classroom training, but has transitioned to online learning Lynn is focusing on Big Data projects and using tools to solve real-world problems Pre-processing and batching data, but not streaming it AWS, Azure, and Google Cloud are all coming out with Big Data-oriented tools Companies need to understand when the market is ready to accept a new paradigm; in the data world, change is more slow than in the programming world If you touch a database and get burned, you are not willing to use it again; or you may have never tried to archive your data; hire a consultant to help you Machine learning APIs give customers value quickly; review them before building custom models Migrating data can be a costly project and restricts where the data lives As Cloud proliferates, how will that impact technical education? Lynn’s Cloud for College Students to the rescue! Shift from interactive to unidirectional, one-to-many learning styles; the Cloud is ready for serverless, but education is not ready for teacherless Road that many of us walked to get to technical skills no longer exists; how to become a modern technologist Ageism: By age 40, you are considered a manager or useless; don’t be afraid to learn something new Links: Digital Ocean AWS Community Hero Microsoft Azure Teachingkidsprogramming.org Digigirlz TKP Labs Lynn Langit on Lynda.com Commonwealth Scientific and Industrial Research Organisation Google BigQuery Amazon Athena AWS Glue Cloud Dataflow Cloud Dataprep Lambda Amazon EC2 Learn Python the Hard Way

Like migrating caribou, you tend to follow the trends of what clients are doing, which dictates what you work on as a consultant. Today, we’re talking to Lynn Langit, an independent Cloud architect. She is an AWS Community Hero, Google Cloud developer expert, and former Microsoft MVP. Lynn is a lifelong learner, and she has worked broad and deep across all three large providers. These days, she works mostly with Google Cloud and AWS, rather than Azure, because that’s what her clients are using. Some of the highlights of the show include: Differences between the West Coast and global use of Cloud Education is key; Lynn is th co-founder of Teachingkidsprogramming.org Lynn helped create curriculum and resources for school-age children; even her young daughter taught classes on how to code Training for teachers was also needed, so TKP Labs was formed to offer fee-based teacher and developer training Lynn started with classroom training, but has transitioned to online learning Lynn is focusing on Big Data projects and using tools to solve real-world problems Pre-processing and batching data, but not streaming it AWS, Azure, and Google Cloud are all coming out with Big Data-oriented tools Companies need to understand when the market is ready to accept a new paradigm; in the data world, change is more slow than in the programming world If you touch a database and get burned, you are not willing to use it again; or you may have never tried to archive your data; hire a consultant to help you Machine learning APIs give customers value quickly; review them before building custom models Migrating data can be a costly project and restricts where the data lives As Cloud proliferates, how will that impact technical education? Lynn’s Cloud for College Students to the rescue! Shift from interactive to unidirectional, one-to-many learning styles; the Cloud is ready for serverless, but education is not ready for teacherless Road that many of us walked to get to technical skills no longer exists; how to become a modern technologist Ageism: By age 40, you are considered a manager or useless; don’t be afraid to learn something new Links: Digital Ocean AWS Community Hero Microsoft Azure

Microsoft has experienced a renaissance. By everything that we've seen coming out of Microsoft over the past few years, it feels like the company is really walking the walk. Instead of just talking about how it’s innovative, it’s demonstrating that. Microsoft has been on an amazing journey, making the progression from telling customers what they need to listening to them and responding by building what they ask for. Today, we’re talking to Corey Sanders, Corporate Vice President of Azure Compute at Microsoft. Some of the highlights of the show include: Customers are asking for Microsoft to help them through support and enabling platforms Storytelling efforts through advocates, who play a double role – engaging and defending Microsoft Customers moving to the Cloud are focused on a continuum and progression; they have stuff to move from one location to another and want all the benefits–better agility, faster startup time, etc. Virtual serial console into existing VMs; this is how people are using this and Microsoft is going to, if not encourage this behavior, at least support it Microsoft is the only Cloud with a single-instance SLA Serial consoles: Windows' has seen less usage, partly due to operational aspects of Windows vs. Linux. It's not a GUI; it's scripting. Does the operating system matter? From a Cloud perspective, it shouldn't have to matter; you should be able to deploy it the way you want Edge enables much more complex and segregated scenarios; that combination with cognitive searches running locally will make it accessible anywhere Branding challenge as customers start to notice that devices are smarter and more complex; will they lose awareness that Microsoft Azure is powering most of these things - they shouldn’t care An awareness of not just what's possible, but what's coming; the democratization of AI Education and fear gap of trying something new and taking that first step; make products and services stupid and simple to use Customers return to add cognitive services and AI capabilities to existing, running deployments, environments, and applications Multi-Cloud solutions can be successful, but there's a caveat; they’re actually built on a service-by-service perspective Azure Stack, offers consistency, but some people may place blame on it for poor data center management practices; some expectations and regulations may be frustrating to some customers, but lets Microsoft offer a consistent experience Freedom and flexibility have been challenges for Microsoft and other products for private Clouds What people need to understand about Azure, including from a dur

Microsoft has experienced a renaissance. By everything that we've seen coming out of Microsoft over the past few years, it feels like the company is really walking the walk. Instead of just talking about how it’s innovative, it’s demonstrating that. Microsoft has been on an amazing journey, making the progression from telling customers what they need to listening to them and responding by building what they ask for. Today, we’re talking to Corey Sanders, Corporate Vice President of Azure Compute at Microsoft. Some of the highlights of the show include: Customers are asking for Microsoft to help them through support and enabling platforms Storytelling efforts through advocates, who play a double role – engaging and defending Microsoft Customers moving to the Cloud are focused on a continuum and progression; they have stuff to move from one location to another and want all the benefits–better agility, faster startup time, etc. Virtual serial console into existing VMs; this is how people are using this and Microsoft is going to, if not encourage this behavior, at least support it Microsoft is the only Cloud with a single-instance SLA Serial consoles: Windows' has seen less usage, partly due to operational aspects of Windows vs. Linux. It's not a GUI; it's scripting. Does the operating system matter? From a Cloud perspective, it shouldn't have to matter; you should be able to deploy it the way you want Edge enables much more complex and segregated scenarios; that combination with cognitive searches running locally will make it accessible anywhere Branding challenge as customers start to notice that devices are smarter and more complex; will they lose awareness that Microsoft Azure is powering most of these things - they shouldn’t care An awareness of not just what's possible, but what's coming; the democratization of AI Education and fear gap of trying something new and taking that first step; make products and services stupid and simple to use Customers return to add cognitive services and AI capabilities to existing, running deployments, environments, and applications Multi-Cloud solutions can be successful, but there's a caveat; they’re actually built on a service-by-service perspective Azure Stack, offers consistency, but some people may place blame on it for poor data center management practices; some expectations and regulations may be frustrating to some customers, but lets Microsoft offer a consistent experience Freedom and flexibility have been challenges for Microsoft and other products for private Clouds What people need to understand about Azure, including from a durability and reliability experience To some extent, scale becomes a necessary prerequisite for some applications Microsoft has taken many steps and is the leader in various areas Links: ReactiveOps Microsoft Azure Corey Sanders on Twitter The Robot Uprising Will Have Very Clean Floors Kubernetes Cassandra Azure Stack

Have you dabbled with IT infrastructure in AWS? Have you been through the process of AWS partnership? Does being an AWS partner add value? Amazon seeks partners that helps drive its business, goals, and value. Today, we’re talking to Justin Brodley, the vice president of Cloud engineering at Ellie Mae. He has been through the AWS partnership process and shares his thoughts about it. He encourages you to find the right partner for your business! Some of the highlights of the show include: Different levels and types of AWS partnerships Shakedown vs. opportunity method for new leads; lead generation expectations Amazon’s improvements eroding business models Partners trying to pivot, but not exclusive to AWS Whether to invest in multi-Cloud Amazon can’t scale its sales team to handle everybody; views partner program as an extension of its salesforce Your company is important and you’re spending a lot of money, but Amazon may not care about you; partner market fills that gap and makes you feel important Corporate prisoner’s dilemma: Your tech company offers something that Amazon doesn’t; but what about when Amazon does offer it? Competitors’ horizontal move to become more diversified Amazon expects partners to offer products and services that it cannot offer yet If partners fail, Amazon decides to do it and do it better Is Amazon’s best interest geared toward its partners or you and your customers? Amazon needs to give incentives and support partners Links: Justin Brodley on Twitter Brodley Group Ellie Mae Digital Ocean AWS Partner Network Lambda API Gateway AWS re:Invent Salesforce

Have you dabbled with IT infrastructure in AWS? Have you been through the process of AWS partnership? Does being an AWS partner add value? Amazon seeks partners that helps drive its business, goals, and value. Today, we’re talking to Justin Brodley, the vice president of Cloud engineering at Ellie Mae. He has been through the AWS partnership process and shares his thoughts about it. He encourages you to find the right partner for your business! Some of the highlights of the show include: Different levels and types of AWS partnerships Shakedown vs. opportunity method for new leads; lead generation expectations Amazon’s improvements eroding business models Partners trying to pivot, but not exclusive to AWS Whether to invest in multi-Cloud Amazon can’t scale its sales team to handle everybody; views partner program as an extension of its salesforce Your company is important and you’re spending a lot of money, but Amazon may not care about you; partner market fills that gap and makes you feel important Corporate prisoner’s dilemma: Your tech company offers something that Amazon doesn’t; but what about when Amazon does offer it? Competitors’ horizontal move to become more diversified Amazon expects partners to offer products and services that it cannot offer yet If partners fail, Amazon decides to do it and do it better Is Amazon’s best interest geared toward its partners or you and your customers? Amazon needs to give incentives and support partners Links: Justin Brodley on Twitter Brodley Group Ellie Mae Digital Ocean AWS Partner Network Lambda API Gateway AWS re:Invent Salesforce Azure Rackspace