InfoQ Homepage News A Team's Transformation from Software Development to ML: Golestan Radwan at QCon NY

The next QCon is in London, Mar 4 - 6, 2019. Save an extra £75.00 with INFOQ75!

: A Team's Transformation from Software Development to ML: Golestan Radwan at QCon NY


As companies start to add Big Data and Machine Learning initiatives to their project portfolios, they face several challenges including the teams' transition from software engineering to data engineering and machine learning. Golestan "Sally" Radwan recently spoke at QCon New York 2018 Conference about her experience in leading a traditional software engineering team on a machine learning/AI journey.

Radwan talked about what went well in the transition of a software architect to data architect. Techniques like frequent and visual communication helped them but they had to make some tough technology and people decisions. She discussed three levels of transformation:

  • Individual
  • Team
  • Business

Individual transformation included moving from languages like PHP and Python; changing the technology was not easy. The developers had to get comfortable with handling completely different data structures, formats, and data sources.

They also had to learn to work a lot more closely with DevOps. She advised the developers to learn fast and be ready to make compromises. It's important to lean on your team. Architects should learn about data and machine learning pipelines. Focus on performance of each algorithm and its pros/cons and suitability for different types of data.

Some of the disciplines and resources they relied on for this transformation included the following:

At the team level, they focused on regular knowledge sharing sessions; "lunch and learn" meetings were held every Thursday, in which team members could share what they learned. They also codifed this knowledge and automated associated processes using CircleCI in order to make the most use of Data Scientists valuable time.

Speaking of Data Science discipline, Radwan suggested to look for some development skills (at least to prototype) when hiring data scientists. Probe their real world understanding to learn how much they care about the big picture, company goals, and how they fit in the organization.

And finally at the organization level, if you have a podium, speak up and set expectations to your stakeholders. Educate and collaborate with other teams. It's important to be clear on targets, goals, and requirements. In the machine learning space, your responsiblility includes privacy, provenance, bias, and quality. Take time to understand the context and implications of what you're doing.

Radwan mentioned the presentation on analyzing the bias in machine learning at 2018 Conference by Rachel Thomas. She concluded the discussion by suggesting the teams resist the urge to overcomplicate things so they can be successful in their machine learning initiatives.

Rate this Article


Hello stranger!

You need to Register an InfoQ account or or login to post comments. But there's so much more behind being registered.

Get the most out of the InfoQ experience.

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Community comments

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p


Is your profile up-to-date? Please take a moment to review and update.

Note: If updating/changing your email, a validation request will be sent

Company name:
Company role:
Company size:
You will be sent an email to validate the new email address. This pop-up will close itself in a few moments.
  • 小龙虾走俏催生新职业“品虾师” 2018-12-15
  • 女子痴迷鹿晗 商场门口对人形立牌拭泪亲吻 2018-12-14
  • 成都:共享办公受追捧 助力写字楼“去库存” 2018-12-14
  • 回复@老老保老张工:伪高工想回到那种你生产的产品再水都有人买单都不会倒闭的日子?没门儿! 2018-12-12
  • 习近平:深入实施创新驱动发展战略 为振兴老工业基地增添原动力 2018-12-12
  • 昆明母婴室地图出炉啦!公众场合喂奶不再羞答答 春城壹网 七彩云南 一网天下 2018-12-10
  • 南昌市新建区司法局深入湖区渔船宣传法律 2018-12-09
  • 银白配色更高贵-热门标签-华商网数码 2018-12-08
  • 超美雾凇冰挂奇观   豫北最大瀑布群变冰帘 2018-12-07
  • 台东“孩子的书屋”:撑起偏乡学童翻转命运的机会 2018-12-07
  • 实验室里“种植”钻石,这样的人造钻戒你能接受吗? 2018-12-06
  • 新赛季CBA联赛常规赛分组 吉、辽、深、广、青同组 2018-12-05
  • 【网络中国节】端午遇上足球杯 平陆交警夜查全力保平安 2018-12-05
  • 法学教育 要离生活更近些 2018-12-04
  • 写“平乐镇”前,小说家颜歌的光怪陆离 2018-12-03
  • 715| 775| 184| 500| 157| 168| 375| 340| 596| 254|