深圳风采预测:Arun Gupta on Managed Container Control Planes on AWS

| Podcast with Arun Gupta Follow 2 Followers by Wesley Reisz Follow 17 Followers on Jul 07, 2018 |

深圳风采开奖号码 Arun Gupta discusses with Wes Reisz some of the container-focused services that AWS offers, including differentiating ECS and EKS. Arun goes into some detail the role that Amazon Fargate plays and goals behinds EKS. Arun wraps ups discussing some of the open source work that AWS has recently been doing in the container space.

Key Takeaways

  • ECS & EKS are both managed control planes; Amazon Fargate is a technology used to provision clusters.
  • ECR is the Amazon Container registry (similar to the Docker Registry).
  • EKS is an opinionated why of running a Kubernetes cluster on AWS. It is a highly available managed control plane available on US East 1 and US West 2 
  • EKS uses a split account. The control plane runs in an Amazon account and the workers run in customer’s account.
  • Upstream compatibility is a core tenet of EKS.

You moved to AWS a few months after Adrian Cockroft. Is there any connection?

  • 2:00 I have known Adrian for some time - we worked together at Sun.
  • 2:05 We were at Stockholm airport, and I was sitting right next to him on the train.
  • 2:20 He said that he was starting a new team up in 2016 and when he had a req he would let me know - and here we are.
  • 2:35 It was all over very quickly - a week or two from interview to offer - I was very impressed with their HR process.

What does a principal open-source technologist at AWS do?

  • 2:55 Adrian has a team under him, focussed on different verticals.
  • 3:05 As a principal open-source technologist I’m focussed specifically on containers and serverless compute.
  • 3:50 My role is to build strategy around open-source for containers and serverless.
  • 3:50 One of the things I helped do when I joined AWS was to join CNCF (Cloud Native Computing Foundation) where I am Amazon’s board representative alongside Adrian.
  • 11:50 I’m responsible for the CNCF strategy at Amazon - I liaison between the different service teams and different products.

What does a typical day look like?

  • 3:45 My job requires hands-on coding, so I do a lot of coding.
  • 3:50 It could be anything from creating a Docker container to spinning up an EKS cluster to setting up an ISDIO or service mesh.
  • 4:05 A lot of it is reviewing the blogs that our partners, customers or SAs are writing.
  • 4:10 We’re an outward facing team and are on Twitter as well.
  • 4:15 We write abstracts for conferences and submitting presentations.
  • 4:20 I go through the abstract is merit-based approved, rather than sponsored.

What’s the difference between ECS, EKS and Fargate?

  • 5:00 ECS (Elastic Cluster Service) and EKS (Elastic Kubernetes Service) are managed services.
  • 5:10 Fargate is only a technology in how the cluster is provisioned.
  • 5:15 In addition, there’s a managed service called ECR (Elastic Container Registry) - like running a private Docker Hub but in the Amazon cloud.
  • 5:25 Those are the four components that you need to understand.
  • 5:35 ECS was launched in 2014 - there was no container orchestration frameworks, Kubernetes wasn’t open-sourced at that time.
  • 5:45 Our customers were asking to have a service where they could orchestrate containers - and so ECS was launched.
  • 5:55 It is successful today - 100s of millions of containers are alunched every day.
  • 6:00 If a customer is all in, they want a deep integration with Amazon WebServices, CloudWatch, CloudTrail and compliance with HIPPA and ECI compliance.

So if you’re not bringing your orchestration, ECS lets Amazon orchestrate your containers?

  • 6:40 Yes, exactly.
  • 6:45 IF you like Kubernetes API and you want Pods and deployments and replica sets, and you want to ensure you don’t have vendor lock-in - to run in the cloud or on private hosted services, then EKS is a good fit.
  • 7:20 The deployment model and API allows you to run on your own Kubernetes clusters or in the cloud.

Where does Fargate come in?

  • 7:35 With both ECS and EKS you get a managed control plane, but you have to bring your own worker nodes in.
  • 7:45 In some cases, you don’t need to wonder about where the compute comes from or how the cluster is provisioned.
  • 8:50 The idea is that you can provision ECS and bring your own cluster nodes from EC2.
  • 8:10 If you don’t care about how the cluster is provisioned, then that’s what Fargate system does.

How does pricing work with Fargate?

  • 8:45 At Re:Invent 2017 we announced ECS and Fargate integration - that’s available today.
  • 8:55 We also announced plans for integration between EKS and Fargate - we can talk about that later.
  • 9:00 When you run ECS clusters where you bring your own worker nodes, you can do on-demand, spot pricing, reserved instances - the prices could vary depending on what you’re using.
  • 9:15 With Fargate, we give you a defined CPU and memory amount with an initial 60s pricing and then per-second pricing after that.
  • 9:30 If you have a burstable load where you want to do something quickly with containers then Fargate might be good for you.

How is Amazon EKS different from upstream Kubernetes?

  • 10:10 One of the core tenets of EKS is upstream compatibility.
  • 10:20 There’s no inferior branches of Kubernetes.
  • 10:30 When you provision an EKS cluster, you get a managed control plane - in either US-East-1 or US-West-2 - and it will give you a highly available control plane with 3 masters and 3 Etcd nodes.
  • 10:45 We have automatic backup, snapshotting of etcd nodes, automatic scaling.
  • 10:55 It’s a lot of undifferentiated heavy lifting that we’re doing for customers.
  • 11:05 You still have to bring your own nodes to the cluster through ECS and attach them to the cluster.
  • 11:10 You can use kubectl from upstream and use that to deploy your pods, manifests or whatever to the cluster.

Are tools fully supported?

  • 11:35 We take an opinionated approach to the control plane, but you can do anything on your compute nodes.
  • 11:40 The worker runs in the customer account, so you can install anything you want on there - but the control plane runs under Amazon’s account.
  • 11:50 Because it’s a managed control plane, we have a set of plugins that we have enabled and are looking for customer feedback.
  • 12:30 Running Kubernetes and managing the cluster is not for the faint-of-heart.
  • 12:35 For example, should you run a single master or three masters? Should the etcd nodes be colocated with the kubernetes masters? How often should you back up or snapshot your etc containers?
  • 12:50 When you are using these with Amazon, what are the best practices?
  • 13:00 A recent survey from CNCF said that 57% of Kubernetes users run on AWS.
  • 13:10 You still have to understand what the AWS best practices are, which is where EKS comes in.

What does bootstrapping an app with EKS look like?

  • 13:30 You fire up an EKS cluster from the console (or an AWS cli).
  • 13:45 When the master is provisioned you can use a cloud formation template which you can use to bring the worker nodes to attach to the cluster.
  • 13:55 You’re using upstream kubectl to control the cluster.
  • 14:00 Typically if you use kubectl it will have configuration to say how to connect to the cluster.
  • 14:05 In AWS you typically use IAM credentials, so there is an additional dependency for performing the authentication mapping using IAM roles.
  • 14:20 Once you’re authenticated, the same kubectl can be used to connect to the application.

What about limitations?

  • 14:50 There’s work to do with EKS - anything we’ve launched on day 1 is a good start, but we always innovate on our customers’ behalf.
  • 15:00 Given the fact that ECS has been out for over 3 years, it helps to having the deeper integrations like CloudWatch and CloudTrail.
  • 15:10 These are some of the items actively on how we can integrate with them - prioritising on customer feedback.

What is the shared responsibility model?

  • 15:30 The shared responsibility model is where Amazon’s responsibility includes looking after the host operating system, the underlying virtualisation layer and the physical security of the data centre.
  • 15:45 Anything on top of that - the guest operating system, the security groups exposed - is the customer’s responsibility.
  • 16:00 For EKS, the control plane is Amazon’s responsibility.
  • 16:05 If you want to upgrade your cluster, it could be a simple check or it could be a manual process - but we’ll take care of the upgrade for you.
  • 16:10 On the worker side: if you want to upgrade the cluster, then it’s the customer’s responsibility - although we’ll provide scripts to help.

What are the success stories of EKS?

  • 16:30 One of the things I like about Amazon is the customer obsession.
  • 16:35 There were about 60 customers that were part of the limited preview which was announced at Re:Invent.
  • 16:45 GoDaddy was a big success story - they released an article about a few weeks back going all in on EKS.
  • 16:55 Myself and the EKS PM were called in to talk about EKS to GoDaddy.
  • 17:15 A few weeks later we found they went with EKS so we were super excited.
  • 17:35 Go to and the customer success stories are published there.

How does EKS and Fargate work?

  • 17:45 EKS and Fargate is available today - we announced that earlier this year.
  • 18:00 What does Fargate mean in the Kubernetes mean?
  • 18:05 Kubernetes is based around nodes, with node agents (kubelet).
  • 18:10 Pods have affinities to nodes and can be deployed on nodes.
  • 18:15 So nodes are a key part of Kubernetes.
  • 18:20 If you look at Fargate, there is no concept of a node.
  • 18:25 Virtual kubelet is a community effort to build something similar to Fargate.
  • 18:40 Virtual kubelet works with a provider, so we have a Fargate provider upstream.
  • 18:50 The key work has to happen in the kubernetes community.
  • 19:00 They are looking at what nodeless means in a kubernetes environment.
  • 19:15 Amazon wants to have upstream discussions and implementation rather than doing things internally.

What are some of the other stories you have at Amazon?

  • 19:30 One of the key tenets of EKS is open-source.
  • 19:35 If you look at how the networking is done with EKS then it’s done with the cni plugin.
  • 19:45 There are about 8-10 cni plugins from various vendors.
  • 19:50 What we did is to build a cni plugin in open-source, and anyone can use it to set up their own cluster with native vpn networking.
  • 20:05 The authenticator we were talking about earlier, to provide IAM credentials to a Kubernetes cluster, was done in open source.
  • 20:15 We talked about virtual kubelet - there is ckws which we are quite active.
  • 20:30 There is CoreOS ingress controller that can be installed with EKS - hopefully sometime in the future we’ll have a prebundled user of CoreOS.
  • 21:00 Last but not least there is a cloud provider working group where we have been working with others to have standard cloud APIs.
  • 21:15 We’re actively hiring for open source kuberentes work.

What’s the story around the service mesh using EKS?

  • 21:30 Because you’re managing the control plane, we define what goes in that based on customer feedback.
  • 21:35 Because you are managing the worker nodes, you can do what you want on them.
  • 21:40 Isdio can be installed on an EKS cluster and will work.
  • 21:50 I was working with a developer advocate earlier on installing isdio on an EKS cluster, to find out what works and what doesn’t, and were filing issues in the open-source as well as making sure it works out of the box on EKS.

What do you have planned for the rest of the year?

  • 22:20 Next week I’m going to be at LinuxCon, ContainerCon, LC3 in Beijing, and an event in Korea.
  • 22:20 Re:Invent planning has already started, we are looking at being at QCon Seattle, meeting a lot of customers around the world and highlighting customer stories.
  • 22:50 I’m particularly passionate about open-source, so if any customers, developers or partners want to blog about open-source on our open-source blog, I’m happy to take feedback.

What are you working on?

  • 23:05 I’ve been interested in service mesh.
  • 23:15 It’s difficult to explain the concept of a pod to a new kubernetes user, but the moment you have the concept of a sidecar automatically injected in the pod, there is an aha! moment.
  • 23:30 Envoy is one part of it, and Isdio, and take that a step forward - that’s what I’m looking at.
  • 23:45 I’m looking at chaos engineering to make your systems more resilient, and how you can build chaos engineering, Isdio and kubernetes together to build resilient engineering.

More about our podcasts

You can keep up-to-date with the podcasts via our RSS feed, and they are available via SoundCloud and iTunes.  From this page you also have access to our recorded show notes.  They all have clickable links that will take you directly to that part of the audio.

Previous podcasts

Rate this Article

Adoption Stage

Hello stranger!

You need to Register an InfoQ account or or login to post comments. But there's so much more behind being registered.

Get the most out of the InfoQ experience.

Tell us what you think

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

typo by Rob Fulwell

"core tenant" should be "core tenet"

Re: typo by Wesley Reisz

Thanks... fixed.

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

2 Discuss

Sponsored Content

Login to InfoQ to interact with what matters most to you.

Recover your password...


Follow your favorite topics and editors

Quick overview of most important highlights in the industry and on the site.


More signal, less noise

Build your own feed by choosing topics you want to read about and editors you want to hear from.


Stay up-to-date

Set up your notifications and don't miss out on content that matters to you

  • 世界杯专版 竞猜型彩票 赔率是公认的竞猜利器 2019-02-21
  • 反制更快更强更准!中国坚决打赢对美贸易自卫反击战! 2019-02-20
  • 北京国际旅博会开幕 旅游产品直降数千提前带热暑期档--旅游频道 2019-02-19
  • 世界杯倒计时:球迷街头狂欢为自己的国家打Call 2019-02-18
  • 这论坛需要风水们创新、发展,他们也只能靠这个了······ 2019-02-18
  • 雷政富狱中发声:否认漏罪举报 不服原判正申诉 2019-02-17
  • 央行:有效防控互联网金融领域风险 2019-02-16
  • 苏57空中姿态控制能力瞬间救场, F22隐身战机无法超越! 2019-02-15
  • 这泼猴总给人萌萌达的感觉 2019-02-15
  • 十年前人大代表工作二三事 2019-02-14
  • 厦门出现不打烊的便民服务站 2019-02-13
  • 故事新西兰强震致民宅破坏严重 超市遭抢购 2019-02-12
  • 中关村医院为中科院科研人员提供就医绿色通道 2019-02-11
  • 候选企业:内蒙古呼和浩特金谷农商银行 2019-02-11
  • 百姓满意的口碑,就是最高荣誉! 2019-02-10
  • 822| 323| 897| 530| 408| 575| 404| 654| 830| 773|