For starters, patching all your PCs might be incredibly hard. Second, we’ll probably see more vulnerabilities in the Intel Instruction Set Architecture (ISA). And third, it’s entirely possible that hackers have been secretly exploiting these problems for years.

So hold on to your hats. In this week’s Security Blogwatch, we overheat some ghosts.

Your humble blogwatcher curated these bloggy bits for your entertainment. Not to mention: Scott Bradlee returns to his roots …

What’s the craic? Ingrid Lunden reports from Lost Wages—patches will come to 90%+ of chips in the next week:

With the microchip processing industry facing perhaps its biggest security scare in its history…Brian Krzanich [CEO] of Intel, took to the stage…at CES to say a few words about the news. … Intel expects to issue updates to its processors soon. More than 90 percent will be getting them within the week, and the rest by the end of January.
…
Krzanich’s words represented a strong shift from the usually upbeat tone of Intel’s CES keynote speeches.

What, if anything, will Intel do differently in future? Mike Rogoway lights the way—Intel reorganizes amid tumult:

Brian Krzanich told employees Monday that he will create a new internal security group. … Krzanich reassigned several top executives to the new organization.
…
Leslie Culbertson will run the new group, called Intel Product Assurance and Security. … Josh Walden, head of Intel's new technology group, will leave that post to work for Culbertson. … Additionally, Krzanich assigned Steve Smith, an Intel vice president … to Cublertson's new organization.

But Thomas Claburn ignites this confession: [You’re fired—Ed.]

After spending last week insisting that the performance impact … "should not be significant," Intel on Tuesday tried to maintain that stance even as it acknowledged SYSmark tests assessing post-patch slowdowns ranging from [2] to 14 per cent. [But] so much consumer and business computing relies on cloud-based servers, which … have exhibited slower response times and increased CPU utilization.
…
Intel's downplaying of meaningful consequences … appears to have become unsustainable after Red Hat … said the impact … ranged from 1 to 20 per cent in its benchmarks and Microsoft … said something similar. … Terry Myerson, president of Microsoft's Windows and device group, did confirm [that] with Windows 10 running on older hardware … "we expect that some users will notice a decrease in system performance."

So how will we install these Intel microcode updates? Will they be included in the Windows and Linux patches? Here’s danieldk:

The microcode update is … lost between reboots.
…
The only way to make microcode updates stick between reboots is with a BIOS/EFI firmware update. … Then the microcode update gets applied during every boot automatically.

Oh brother. Does that mean IT has to update the firmware in every PC? Bruce Schneier pontificates depressingly:

Some of the patches require updating the computer's firmware. This is much harder to walk consumers through, and is more likely to permanently brick the device if something goes wrong.
…
In November, Intel released a firmware update to fix … another flaw in its microprocessors. But it couldn't get that update directly to users; it had to work with the individual hardware companies, and some of them just weren't capable of getting the update to their customers.
…
Some patches require users to disable the computer's password, which means organizations can't automate the patch. Some anti-virus software blocks the patch, or -- worse -- crashes the computer.
…
These aren't normal software vulnerabilities. … These vulnerabilities are in the fundamentals of how the microprocessor operates. … Security researchers are starting to look more closely at these systems, so expect to hear about more vulnerabilities along these lines.
…
More is coming -- and what they'll find will be worse than either Spectre or Meltdown. … These will be similarly impossible to fix, and the only strategy will be to throw our devices away and buy new ones.

Sigh. Is it even worth it? Here’s kev009:

This new ucode changes fencing and branching prediction semantics.
…
While most people should run the new ucode and pending kernel and toolchain fixes, not all must. Most businesses buy computers on rated performance, and they are about to take an unexpected performance haircut.

Okay, but can we just step back and look at the big picture for a moment? Here’s tptacek:

Can I put a plug in again for how ****ing cool the Meltdown and Spectre attacks are? They're much more interesting than just cache timing, which … have been well-known for at least a decade.
…
* Meltdown and Spectre involve…instructions that from the perspective of the ISA never actually run.

* Spectre v1 undermines the entire concept of a bounds check. … There might not be a clean fix! Load fences after ever bounds check?

* Spectre v2 goes even further than that, and allows attackers to literally pick the locations target programs will execute from. … And look at the fix to that: retpolines? Compilers can't directly emit indirect jumps anymore?
…
It's good that we're all recognizing how big a problem cache timing is. … But Meltdown and Spectre are not simply cache timing vulnerabilities; they're a re-imagining of what you can do to a modern ISA by targeting the microarchitecture.

“May you live in interesting times.” Kevin Beaumont brings us this important information:

It’s … my first month … as a security vulnerability manager at my new job. Hello. It has been an interesting week.
…
Last week, Microsoft issued January’s cumulative security fixes. … These updates came with many caveats, and the Microsoft knowledge base articles have had extensive edits since publishing.
…
On Windows Server, the Meltdown and Spectre patches don’t actually do a thing. … Unless you actually add [three registry] keys the patches don’t actually enable the CPU mitigations.
…
My belief is organisations shouldn’t rush these patches out. They need to carefully test and see where they need to mitigate the vulnerability.. … Organisations need to carefully assess and manage their situation.

Especially if their PCs run AMD CPUs. Amirite? All you need to know is in Tom Warren’s headline—Microsoft halts AMD Meltdown and Spectre patches after reports of unbootable PCs:

Microsoft has paused distributing its Meltdown and Spectre security updates for some older AMD machines after reports of PCs not booting. … Microsoft is blaming AMD’s documentation for the unexpected problems.

How have we not known for ages about problems with speculative execution? Andy Greenberg wonders if perhaps we have:

Security researcher Anders Fogh, a malware analyst for German firm GData, in July wrote … that he had been exploring a curious feature of modern microprocessors called speculative execution. … Perhaps, Fogh suggested, that out-of-order flexibility could allow malicious code to manipulate a processor to access a portion of memory it shouldn't have access to.

July 2017? That’s nothing, according to Steve Gibson:

What's Old is New Again ... 1995: "The Intel 80x86 Processor Architecture: Pitfalls for Secure Systems." … on page 9 under section "4.2 Security Flaws": Item 6. Prefetching may fetch otherwise inaccessible instructions in Virtual 8086 mode. (From a paper in 1992!) -- 25 years ago.

1992? That’s nothing, according to Antique Geekmeister:

I'm following an intriguing discussion of similar side-channel attacks on Multics systems on GE hardware in roughly 1970. It's not a new problem. I've been trying to explain repeatedly to some colleagues while reviewing these attacks that doing "speculative compilation" is very appealing at first glance, but the work involved in doing it is not free. Security risks and maintenance of the resources are critical and related costs of such optimization.

The moral of the story? Don’t expect things to get better in 2018. This could be the tip of an extremely cold iceberg.

And finally …

You’ve Got a Friend in Me

You have been reading Security Blogwatch by Richi Jennings. Richi curates the best bloggy bits, finest forums, and weirdest websites … so you don’t have to. Hate mail may be directed to @RiCHi or sbw@richi.uk. Ask your doctor before reading. Your mileage may vary. E&OE.

Image source: Alexandra (cc0)

Next

1.4B leaked passwords in 41GB dump: Stop the madness!

Richi Jennings

Dec 14, 2017

Next

Cloud database security cleanup on aisle 4: Alteryx breach a capper for 2017

Richi Jennings

Dec 21, 2017

Schwab had also considered Amazon EC2 Container Service (ECS). “But for us, Kubernetes feels much more like a framework, and that means we can pick the parts we like and not use the parts we do not like,” he says. That said, moving forward with Kubernetes has been a journey with a learning curve.

It's been 10 months since deployment. Here what Schwab's team has learned along the way.

A journey to Kubernetes begins

PhraseApp sells a software localization platform, built using Ruby on Rails, that lets users ship software in different languages faster, and with less effort. The application, also called PhraseApp,  has multiple dependencies in terms of system packages and libraries. Containers now provide the immutable infrastructure required to ensure the greatest robustness and efficiency for the application architecture and meet the high-agility demands of supporting multiple source code deployments per day, Schwab says.

PhraseApp and its database began life on a bare-metal server at a German hosting company, where Ruby on Rails and its system packages were manually installed. Schwab's team used Capistrano, the open source remote server automation tool, to migrate PhraseApp and its database to AWS in early 2013, and used golden master Amazon Machine Images (AMIs) to horizontally scale up its application servers.

The team then stepped PhraseApp up to Docker to deal with a few remaining challenges, such as slow deployments. With Docker, Schwab could maintain the OS, packages, libraries, and code in one image to reduce deployment times.

"The container model helped us because we now only need to provide Docker images for the application, whose service-oriented architecture encompasses log indices, metrics, offsite backups of its database, and so on," he says. “We no longer need to package these, for example, as Linux packages or provide some other reliable way to deliver them. We have separate repositories in our Docker Amazon EC2 Container Registry (ECR) and run all these services in containers.”

The PhraseApp team initially used a custom infrastructure management tool, an internal server cluster scheduler the team dubbed Wunderproxy, to create Docker images and new containers by way of APIs. While this led to faster deployments and simpler rollbacks, Schwab's team wanted to get away from having to support its own custom controller management code and being limited to a legacy version of Docker.

Kubernetes vs. Amazon ECS

Early last year Schwab began evaluating Kubernetes and Amazon ECS to see which would better serve as a highly scalable system for PhraseApp’s container management requirements.

“When we evaluated ECS, it still felt a bit immature,” Schwab says, describing incidents such as the ECS agent crashing on its nodes for no apparent reason. The AWS Elastic Load Balancing (ELB) Classic Load Balancer (CLB) also had limitations, such as not supporting the ability to run multiple containers of the same kind on a single host.

“But the biggest issue we had was the lack of transparency,” he says, noting that ECS is closed source and “felt like a black box.” The PhraseApp team found itself dealing with many broken requests due to time-outs, but there was no way for it to determine if that was due to the ELB or ECS itself.

Although PhraseApp’s cloud version lives on AWS, its developers evaluated Kubernetes on Google Container Engine. Things started off on the right foot because the group did not have to set up any parts of its cluster itself, which had been a requirement of ECS at the time of its testing.

That meant the product team could focus more on implementing its continuous delivery pipeline for PhraseApp, Schwab says. Adding to its appeal was the fact that Kubernetes didn’t force-feed the developers capabilities they didn’t want, such as using Ingress controllers to handle incoming traffic to the cluster servers.

“ECS gave us more of a ‘this way or the highway’ feeling, that we were forced to use it as the people at AWS thought it should be used. Kubernetes gives us quite a lot of options that help us solve the challenges we need to solve, like running cronjobs and asynchronous jobs.”
—Tobias Schwab

Today with Kubernetes, the PhraseApp cron jobs can be distributed in the whole cluster instead of using a dedicated host where things would slow down whenever multiple jobs were running. And it can deploy the same codebase for both its front-end API and worker processes for long-running web requests in sync. In fact, Schwab says that the most important test it ran to decide between Kubernetes and ECS was to deploy the front-end API component of PhraseApp on both architectures.

Another option available is to run PhraseApp services such as the In-Context Editor application—where translators can edit the copy of a website through an overlay interface while browsing that website—in the Kubernetes cluster, too. “We also provide an on-premises solution of PhraseApp which is already based on Docker,” Schwab adds. “Kubernetes might also give us the option to provide a much easier setup.”

PhraseApp, Kubernetes go to production

Earlier this year, Schwab's team finally deployed PhraseApp with Kubernetes. Between the evaluation that began early last year and that step, the team also committed to kc, its generic tool to build and deploy applications on Kubernetes, including the business-growth application its sales and marketing team uses to gather data about customers and trials.

“It was the first client of our AWS-based Kubernetes setup while we learned how to run our own cluster,” Schwab says. His company, he adds, likes to first introduce new technologies in ways that don’t directly impact customers, in case any issues arise that would affect their operations.

“This also allows us to move much faster in these areas, compared to the caution we would need to take if we migrated our production system first,” he says. Additionally, it’s a good way to build the same amount of confidence in the brand-new Kubernetes deployment that it had in its previous architecture, which was rock solid and hardly ever gave the organization problems.

It was invaluable, Schwab says, to have time to gain experience by using its growth app to build trust in Kubernetes itself; to determine how to set up its clusters (which changed a few times in the process and which Schwab says was definitely the “biggest pain”); and to shape its ideas about how it would use the container management platform.

The big wins

Schwab calls the company's experience since February, when it deployed PhraseApp with Kubernetes 100% positive. "We never had any production issues which were related to the Kubernetes cluster at all.”

The gains included:

• Not having to think about how to deploy and configure a specific service (such as one for monitoring its SQS queues), especially in combination with the kc tool.
• Being able to quickly test specific changes in the codebase before deploying them to end users.
• Doing away with configuration management updates. Instead of changing already-running services, it can create new containers and terminate the old ones.
• Having a fully transparent infrastructure, with the ability to get lists of the services and exact configurations needed to run all aspects of its application.
• Driving better resource utilization. It can deploy many services on a single node without have them interfere with one another
• Having the scalability to launch new nodes into its cluster in less than ten minutes, and remove them even faster.

Taking Kubernetes to the next level

PhraseApp's plans include moving from running the cluster with kubeadm, part of Kubernetes, to the kops opinionated provisioning system for AWS. That would bring to the table things such as independent node groups, nodes running in autoscaling groups, and the possibility to update the Kubernetes version more or less for free, Schwab says.

Kubernetes gives the company a tool to move PhraseApp away from a monolithic architecture to one that is more microservices-oriented, with reasonable, rather than excessive, operational overhead, says Schwab.

“The effort and overhead to deploy a microservice architecture would have been way too high for us if we had wanted to deploy it without a cluster scheduler like Kubernetes,” he says. “We do not have a dedicated IT operations team, but we try to ‘live DevOps,’ focusing instead on delivering features to our clients and then on maintaining our infrastructure.”

Kubernetes lessons from the trenches

If you're wondering whether Kubernetes is the right container management platform for your organization, Schwab has a few recommendations. 

• If you’re starting a new project, use the Google Container Engine cluster manager.  It is “by far the simplest and most robust way to deploy a Kubernetes cluster in production,” he says.
• If you’re already bound to AWS or prefer AWS over Google Cloud, though, use kops or work with a service provider that can help you set up and run the cluster for you.

“Kubeadm is already a really nice tool, but we would like to have somebody who manages our cluster for us, and provide us with a highly available master and a robust way to upgrade the cluster,” he says.

Schwab acknowledges that dedicated Ops teams can probably be trained to run and maintain a Kubernetes cluster. But, he advises: 

• Bring in external consultants to get your Ops team up to speed.

Finally, Schwab says, join the Kubernetes Slack community. “You will find a lot of people who are willing to help you with any kind of problem.”

To help you review the various ecosystem changes that happened last year and get ahead of the curve on the trends for 2018, TechBeacon interviewed experts in architecture, microservices, data science, Java, JavaScript, and Android. Here's what we learned. 

Expect a greater focus on evolutionary architecture and microservices data distribution

“Evolutionary architecture and automation of architectural governance and analysis are going to be a major focus for software firms in 2018," said Mark Richards, author of Software Architecture Patterns. To get up to date on the latest software architecture trends, he recommends reading Neal Ford's Building Evolutionary Architectures.

It isn’t just about pushing for a switch to microservices—although microservices do qualify as an evolutionary architecture. It's about focusing on the development of architectures that can handle significant change over time. As long as the architecture has …

• Modularity and loose coupling
• Organization around business capabilities
• And the ability to conduct operationally inexpensive experiment

... it qualifies as evolutionary architecture. It doesn’t have to be microservices.

As for the evolution of microservices, many organizations have already figured out how to make them, Richards said, but data distribution is still a difficult aspect of breaking down a monolithic architecture, and there's no easy way around that.

He expects more organizations to hit this snag in 2018. “Most companies have figured out the functional and technical portion of microservices now,” he said, “but the hard part of microservices is still data."

Serverless will continue to grow, with AWS Lambda dominating

Bradley Holt, developer advocate and senior engineer at IBM, said 2018 is the year that serverless will gain a foothold with developers. “Functions as a service (FaaS) allowed developers to compose applications and services from fine-grained components with the ability to scale capacity and cost based on demand,” he said.

As the number of serverless and FaaS resources and conferences grows, so will developer interest. As it stands, AWS Lambda is the go-to serverless platform for most developers. And even though serverless technology is still new, AWS will continue to dominate the serverless space for 2018, just as it dominates in other cloud infrastructure areas.

Almost all companies will ask if they need a data scientist

“Developers are beginning to realize the possibilities surrounding data science and ‘cognitive’ technologies such as machine learning, artificial intelligence, and natural language processing,” said Holt. “I predict increased collaboration between application developers and data scientists, and further utilization of machine learning and related technologies.”

Richards agreed that "data science is everywhere now,” as evidenced by the growth of R (a statistical programming language) and Python's data science libraries in 2017. That growth will continue into 2018, driven by the hype around artificial intelligence (AI).

Java will be more vibrant than ever, with a new release cadence

Martijn Verburg, a leader of the London Java Community, said 2017 was a monumental year in Java's evolution. 

With Java SE moving to a six-month release cycle, Oracle's JDK components migrating to the open-source OpenJDK, and Java EE moving to Eclipse, Java will only get better, Verburg said. “We're going to see Java SE released more often, with more features.” And because OpenJDK will be the same as Oracle's JDK, developers won’t face licensing issues.

These changes come in the wake of a major controversy that happened last year, when a majority of the Java Community Process executive committee voted against Project Jigsaw, a proposed implementation of Java modules. Verburg and others worried that Jigsaw would harm the industry, so they asked for modifications.

“Mark Reinhold and Oracle—to their credit—listened to us and made the changes,” said Verburg, and the new proposal passed.

Shortly after that, Oracle proposed that the OpenJDK move to twice-yearly releases for new Java versions, a measure the community welcomed. Slow releases—often taking a year or more to complete—were a common criticism of Java. So this year, the community may see fewer posts arguing that Java is dead.

“I expect 2018 to be a year of innovation for Java as almost all of the remaining legal, IP, and release delay issues have now been resolved.” —Martijn Verburg, CEO of JClarity and a key contributor to OpenJDK

Java EE (now EE4J) will go cloud-native

Oracle’s donation of Java Enterprise Edition (Java EE) to the open-source Eclipse Foundation, which Eclipse renamed EE4J, was another huge change for Java in 2017.

The Eclipse Foundation also launched MicroProfile, a platform definition that optimizes EE4J for microservices. Together, EE4J and MicroProfile will enhance traditional Java EE APIs in 2018, turning them into cloud-, asynchronous-, and microservice-compatible APIs, while following a semi-standardized process.

The MicroProfile community doesn’t want to take a hard standards approach, according to its website, because that would slow down innovation. Instead, it wants to allow projects within MicroProfile to innovate and consider standardization only when contributors agree on approaches.

MicroProfile should help more enterprise Java developers move to microservices—specifically those who still prefer Java EE/EE4J implementations (for example, CDI) over Spring framework ones (like Spring DI). But developers who don’t mind using the Spring ecosystem can use Spring Boot, a microservice-building framework that has been available to Java developers for several years.

Java 10 will include major performance boosts

Last year, Verburg predicted that Java 9 wouldn’t have as much impact as Java 8, and so far that has indeed been what happened. Java 9 comes with big changes that could be slowing adoption, since it forces developers to update some of their applications in order to use the module system. The more frequent language releases coming in 2018 are another reason adoption has been slow. Since Java updates now come twice a year, some teams may decide to skip a few versions before catching up.

But Java 10, with its many performance improvements, will be harder for developers to skip, Verburg said. Here are his highlights for Java 10:

JEP 286: Local-Variable Type Inference—This represents a massive improvement in how developers use switch statements, instanceof operators, and so on. It will make Java code clearer and more concise.

JEP 296: Consolidate the JDK Forest into a Single Repository—Eases OpenJDK development.

JEP 304: Garbage-Collector Interface—Makes it easier for Shenandoah and other garbage collection improvements to come into the platform.

JEP 307: Parallel Full GC for G1—A much-needed performance boost for garbage collection.

JEP 310: Application Class-Data Sharing—Another performance-enhancing feature.

JEP 312: Thread-Local Handshakes—A precursor to a lot of interesting optimizations around performance.

JEP 313: Remove the Native-Header Generation Tool (javah)—Removes the javah tool because javac, which serves a similar purpose, has superior functionality.

JEP 314: Additional Unicode Language-Tag Extensions—Implements more of the Unicode extensions specified in the latest Unicode Locale Data Markup Language (LDML) specification, in the relevant JDK classes.

JEP 316: Heap Allocation on Alternative Memory Devices—Extends where Java can run.

JEP 317: Experimental Java-Based JIT Compiler—Preparation for the Graal project, which will make it feasible to write a programmable, optimizable runtime in Java.

JEP 319: Root Certificates—Enhances security.

JEP 322: Time-Based Release Versioning—Moves Java back to a sensible versioning scheme.

As for Java 11, expected before the end of the year, OpenJDK developers haven't confirmed any features. But Verburg thinksOpenJDK developers will begin building the foundations for introducing value types in Java 11.

Kotlin will get even more support from Android

As predicted, the Kotlin programming language took off last year. Since becoming an official language for Android apps at Google I/O, its use among Android developers has grown significantly. Android developer and TechBeacon contributor Aritra Roy expects Kotlin to continue gaining traction in 2018.

“There is a lot of official documentation that is still in Java, which will slowly start getting converted to Kotlin,” he said. He also expects Google to invest more resources into the Architecture Components library.

“One more thing that is going to be big on Android in 2018 is AI,” Roy said. “More AI capabilities will be available directly and natively on the Android OS itself so that any app developer can start leveraging them.”

Expect evolution for CSS and steady improvement in JavaScript

JavaScript isn’t going to have an exciting new feature every year, but at least it’s adding features annually, said Ryan Lewis, software engineer at Kuali and a Node.js contributor and speaker.

“The ECMAScript 8th edition [ES2017] release didn't contain many big new features [watch the full overview here]. The main feature was async/await, the next evolution of Promises,” he said. The release also includes a few small additions to the API, such as new functions.

It was a good thing that async/await gained broad acceptance, especially on Node.js, said JavaScript speaker and trainer Dr. Axel Rauschmayer. He also likes the tooling additions to the JavaScript ecosystem, such as Prettier, which brings automatic source code formatting to JavaScript.

The features in the next version of JavaScript (ES2018) are still in flux, Rauschmayer said, but he’s excited about these potential features:

• Rest/spread properties
• Asynchronous iteration
• Dynamic import() of modules
• Named capture groups for regular expressions
• BigInt (arbitrary-precision integers)
• Private fields for objects
• New array methods: .flatMap() and .flatten()

The changes in JavaScript/ECMAScript are starting to ramp down after a few years of abundant new features, said Trey Huffine, founder of Gitconnected, but he expects CSS to take a major leap forward in 2018.

“We'll see a rapid evolution in CSS much like what JavaScript experienced two or three years ago,” he said. “With JavaScript stabilizing, and developers almost unanimously agreeing that component architecture is the best way to build modern web applications, the front-end community will try to address the issues with CSS and work toward a general solution that fits well with the way we're architecting projects.”

WebAssembly could also have a large impact on the JavaScript ecosystem. “While I'm unsure of the impact, I believe we'll see WebAssembly talked about and used much more in 2018,” Huffine said. “It offers a way to yield drastic performance improvements for the web.”

Need to catch up on other major changes in front-end development in 2017? Huffine provides a great recap here. 

Node.js will withstand controversy, improve performance

Node.js continues to stay on track, with twice-yearly releases, including one long-term service release annually. That's a big benefit for enterprise adopters. Node is keeping up with most of the features in ES2017, and the Node.js package manager (NPM) has had many updates.

In 2016, Facebook released its Yarn Node.js package manager, which was so good it threatened to usurp NPM in popularity. At the time, Lewis thought that NPM wouldn’t get pushed aside, but instead would adopt Yarn’s best features.

It turns out he was right. In 2017, NPM released a major update that gave it parity with most of Yarn’s features.

Rauschmayer, however, still has one item on his 2018 NPM wish list: “that the community will establish clearly defined best practices for publishing NPM modules created via compiler-to-JavaScript languages, such as Babel or Typescript.”

It wasn’t all rainbows and unicorns for Node.js last year. As serverless platforms built on Node.js have become more popular, keeping up with regular security patches has become even more critical for cloud service providers.

“As Node.js spreads to different corners of the web, upgrading becomes difficult,” said Lewis. “A perfect example is the embedded Node.js engines on serverless function services like AWS Lambda and Google Cloud Functions.” Your cloud service provider must upgrade those Node.js versions when new security flaws are found, and they might not always do that so quickly. "Google upgraded its Node.js version for Cloud Functions within a week after the most severe vulnerabilities came to light last year, but Amazon took months,” he said.

Also, Node.js community issues resulted in the dissolution of the Node.js Core Technical Committee (CTC) in 2017. “This resulted in some bad blood in the community, but also some positive results,” Lewis said. “The moderation team has been formed, which I'm the temporary chair of, and we've been working to clarify and standardize how moderation activities occur.” The goal, he said, is to ensure that the Node.js community is a safe space for everyone to discuss technology and contribute to the platform.

The biggest news for Node.js was the introduction of support for ES6 modules (modules standardized in the ECMAScript/JavaScript language itself) in Version 9. “This was a huge decision, with many detractors arguing about the correct implementation of ES6 modules,” Lewis said. “Node.js already had a module pattern, following the Common.js pattern, so making ES6 modules work with Node.js was a large task.” The main implementation mechanism is a new file extension (.mjs) to designate ES6 module files. “Many see this new file extension as fragmenting the Node.js ecosystem.”

Rauschmayer said he hopes that ES modules will finally work seamlessly across platforms. “For Node.js, the transition to ES modules is much harder than for browsers because its module system works synchronously,” he said. “But it is slowly getting there.”

Node.js will see two new releases in 2018: 10 LTS and 11. Lewis doesn’t expect many new features, but plenty of performance improvements will come from the continuing improvements to V8.

React reigns today; ReasonML could be next

React still maintains its place atop the market-share charts for front-end frameworks, according to the recently released State of JavaScript survey. Angular’s popularity continues to decline, while Vue's small following continues to grow (especially in China). React Version 16, released last year, increased performance by rewriting the framework internals while keeping the API consistent.

But despite these successes, the reigning front-end framework had a crisis last year when the Apache Software Foundation put React and other Facebook-produced software on a list of projects it didn't consider open source due to the licensing models. Others in the open-source community agreed with Apache, and Facebook decided to remove the contentious clauses from its licenses.

React is now well positioned to dominate in 2018 and beyond, and Rauschmeyer is bullish about other web development technologies coming out of Facebook as well.

“I’m excited about ReasonML, a new object-functional programming language,” he said. “I like its clean design based on the OCaml programming language and its emphasis on good interoperability with JavaScript. I wouldn’t use it in production yet, but it will be ready soon.”

Facebook released ReasonML in 2017, and Axel is anticipating more open-source releases from Facebook this year. “It will release new offerings in areas they haven't touched yet, such as Node.js web frameworks,” he said. “They continue to present their own perspectives on popular libraries—such as React for front-end, Jest for testing, and Yarn for package management. The team there seems highly motivated and has lots of ideas.”

Key takeaways and next actions for 2018

2018 will be filled with plenty of twists and turns for developers, including revolutionary new technologies, game-changing updates to existing ones, and certainly some controversies. Here are the things that will have the biggest impact on you as a developer 2018:

• Finding companies you trust to handle security concerns that affect you will be a critical challenge.
• Having found that public pressure works, even on tech giants such as Facebook and Google, the developer community will put more pressure on organizations that aren't creating an inclusive environment.
• Expect a huge push to adopt modularity, as both Java and JavaScript refine support for modules.
• More organizations will be investigating whether data science and machine learning can help their business, and you should too.
• Developers will find more even ways to benefit from using serverless platforms.

What are the trends that will most affect you or your development team in 2018? Post your thoughts below.

To my surprise, I discovered six key advantages.

1. Shared resources: Identifiers and more

Unreliable web element locators are a common cause of flaky test automation. As developers change identifiers within an application, the test code gets out of sync and must be updated. When you create a new automation project, the first benefit is that you can take advantage of libraries that define those identifiers.

For example, developers have a class that holds the accessibility identifiers used on all elements throughout the application. So, instead of going through the painful task of digging through the Document Object Model (DOM) to construct reliable identifiers, only to have them inevitably break when the application changes, you simply use the same identifiers that the developers use. As you make updates to those identifiers, the automation code remains stable because it points to the same source.

This was such an epiphany for me that I dug around in the application code to see what else I could use to make my automation code more robust. One developer pointed out that the library of localized strings can also provide value. Yes, it certainly can!

In many tests, you use assertions to verify text. The vast majority of automation projects I've worked on were scoped to English-only because trying to target multiple languages would require more effort than management was willing to invest—for test code, that is. But for application code, localization is a must.

By living in the same repository, our test code can run against any of the languages we support, with minimal effort. The only thing we needed to do to achieve this was to avoid hard-coding text within the test code. Instead, we use the library of localized text that already exists in the shared repository.

2. Access to lower layers

Most practitioners agree that automating tests at the UI layer is costly. Because of this, I'm always looking for shortcuts within the application to test functionality at a layer lower than the UI.

Co-locating your automation code with your application code makes this task much easier. Not only am I able to use web services for faster, less brittle execution, but I'm able to call into business methods that may not have public-facing APIs.

It's simply a matter of figuring out which function is being called from the UI. Then you can skip the UI and call the function directly. It also helps make your test code more reliable.

3. Earlier feedback

Most developers are good about running unit tests before checking in code. After all, no one wants to break the build. As more teams move toward continuous integration/deployment, though, they need more thorough test automation to gate the deploys.

When your test automation code lives in a different code repository, you typically check in and build the application code before executing the test automation. But when I had my automation code in the same repository, I saw developers execute the longer automated scenarios at the same time that they ran their unit tests—before check-in. This gave them much earlier feedback on the code that they wanted to submit and put our team a step closer to its continuous deployment goals.

4. Developers become more invested

When developers break unit tests before check-in, they typically evaluate the test to see if the failure is due to an intentional change. If it is, they update the unit test to reflect the new intention of the application. If it isn't, they fix their code.

Either way, code should not be checked in until it's working as intended and all unit tests have passed. When your automation code lives in a different repository, developers are less likely to make code fixes for the test automation when needed. But if these tests are located in the shared repository and are run as part of the earlier feedback loop, developers are more likely to help with maintenance.

Also, when automation engineers are checking code into the shared repository, our developers are more likely to participate in code reviews of the automation code. The dividing line between "their" code and "our" code becomes much thinner, and developers feel more invested.

5. Automation engineers become more involved

Just as the developers help more with the automation and application code are co-located, it's also easier for automation engineers to help more with unit tests when they are comfortable with the shared repository.

While developers primarily carry the load of writing unit tests, automation engineers, who tend to be more skilled at testing, may have additional scenarios that they want to cover at the unit test layer. Instead of asking the developer to add them, automation engineers can simply add the unit tests themselves. This further blurs the line marking who owns test versus automation code, and instead reinforces a culture where quality is everyone's responsibility.

6. Application code becomes more testable

As I became more comfortable adding unit tests, using shared resources, and calling into the business logic layer, I realized my favorite benefit of all: I no longer have to deal with an application that's not testable.

In the past I'd have to beg management to free up developers to work on making the application more testable, and when that was unsuccessful (which was about 95% of the time), I'd have to jump through hoops in my automation code or abandon my test altogether.

But by having direct access to the application code, familiarity with the project, and a comfort level sufficient to make changes, automation engineers can implement the changes needed to make testing easier. Examples include everything from disabling random, intermittent popups (e.g., "Is your phone number still valid?"), which are absolute nightmares for automation; to including useful logging that helps test code verify more detailed and nuanced background events.

Strangely enough, making direct changes to the application code is also a sure way to guarantee developers' participation in the code review. They can see the struggles you have with test automation, and learn ways to produce more testable code themselves.

Co-mingle happily

Those were all of the wonderful benefits I missed by insisting that all of my automation projects reside in their own repositories. I'm glad I challenged that belief and opened myself up to new experiences, and you should try it too. If you have contributed to a codebase that's shared between application and automation code, I'd love to hear about your experiences in the comments below.

Go see Angie Jones' session, "Which tests should we automate?" at the TISQA conference in Chapel Hill, North Carolina, February 27-28. And although the Automation Guild conference is over, you can still purchase videos of the entire online conference, which includes Jones' roundtable discussion.

To get a glimpse of what the year ahead holds for IT operations management, we asked IT leaders what trends they see developing. The top responses below represent the issues most frequently mentioned (although it's by no means a canonical ranking of concerns).

1. The skills gap will grow wider

The skills gap in IT is only going to get worse. According to TechnologyFirst.org, the US creates 120,000 new jobs that require a degree in computer science each year, but the US educational system produces only 49,000 related degrees. That's a shortfall of 71,000 computer science professionals annually.

The result is that employers will continue to struggle to staff their IT departments. Acorrding to a study by analytics firm Burning Glass Technologies and CompTIA, some 558,713 IT jobs remain open 90 days after posting.

There’s no indication the situation will improve in 2018. More likely, it will get worse. BitTitan CEO Geeman Yip says that the President's executive order earlier this year directing federal agencies to propose changes to the H-1B visa program, which has allowed companies to hire skilled foreign workers for jobs they've been unable to fill with Americans, will make it even more difficult to secure data scientists, engineers, developers, and other IT talent.

The result? Companies may have no other choice but to farm out their most skilled operations, says Mark Kirstein, Yip's colleague at BitTitan.

"Lack of talent continues to inhibit cloud adoption, which means CTOs without the tools to complete more technical tasks are shipping them over to their service providers. Particularly around security, expect service providers to beef up their technical talent and develop accessible how-to programs for their customers."
—Mark Kirstein, VP of products, BitTitan

2. IT Ops will be more tied to end-user experience

As more daily interactions become digitized, consumer expectations for a seamless experience are growing—and companies aren't keeping up.

In its inaugural State of Digital Operations Report, PagerDuty surveyed 300 IT personnel in development and operations, and more than 300 consumers. It found that nearly 70% of consumers abandon an unresponsive or slow digital app or service in 15 minutes or less, but more than one-third of IT respondents said it takes an average of 30 minutes or longer to resolve such issues.

The report also uncovered a reality-perception gap among IT professionals: While nearly 84% of IT personnel said they are confident that their organization is well-prepared to support digital services, more than half said they experienced incidents that affect customers, such as slowness or downtime, at least once a week.

As the role of IT Ops continues to grow more closely tied to end-user experience, it will change the nature of IT Ops' relationships with other lines of business—particularly customer service, said PagerDuty's head of DevOps, Eric Sigler.

The pressure is now on IT operations to address service disruptions quickly while keeping customer support in lock-step as it resolves problems. As we've seen with recent headline-making data breaches, Sigler says, this is particularly true in the case of security incidents.

"Real-time customer communication about IT and security issues is becoming table stakes, and IT Ops will need to reconsider their business-wide communications strategy in order to meet these expectations."
—Eric Sigler

3. There will be more and bigger security breaches

You're not having déjà vu. Security is a perennial top IT concern, as highlighted by a pair of high-profile breaches this year. Last July, data for as many as 14 million Verizon subscribers was left available for download on an unsecured Amazon server that was controlled by a third-party vendor. Then in September, Equifax announced that personal data of about 143 million American consumers was exposed in a data breach. That number was revised upward to 145.5 million as of October.

The incidents were another reminder of the woeful state of security practices across many organizations.

"A lack of resources allocated to security training—both technical and general awareness—increases the risk a company faces related to technical debt, human error, and consistent adoption of security best practices by personnel across the enterprise," said David Buchanan, IT security and assurance partner with Delap.

"Technical debt continues to pose a concern as teams pressured to rapidly prototype and release solutions on tight timelines increasingly bypass controls and security best practices for the sake of meeting release dates."
—David Buchanan

"Equifax is not a 'unicorn,' as most enterprises don't have a comprehensive security assurance program in place. And if they do, it's often composed of manual efforts,” said Mike Kail, CTO of Cybric.

"In order to keep up with the velocity of development, automation and orchestration need to be leveraged."
—Mike Kail

4. IoT will become a workplace security threat

While this year hasn't seen any headline-grabbing Internet of Things (IoT) incidents on par with ones such as the malware-driven distributed denial-of-service attacks that took down Twitter, Spotify, and PayPal last year, the likelihood that another will occur increases every day. IoT tecnology will be in 95% of new electronic product designs by 2020 and new threats will continue to emerge through 2021, according to Gartner.

While the home has been the primary domain of most IoT devices, they are increasingly found in the workplace, where everything from connected espresso machines to printers and copiers to conference-room displays to digital signage provide access points for hackers. So going forward, having full visibility into your IT environment will be crucial for implementing an effective security strategy, Delap's Buchanan said.

The security of many IoT devices relies primarily on the host network, so security teams must harden those. They also must implement mitigating controls to actively protect IoT devices, many of which are currently insecure, providing hackers with an easy entry point into the corporate network, he said.

5. Blockchain will blow up

"Services based on blockchain are going to break out in 2018," said Richard Stiennon, chief strategy officer of Blancco Technology Group. "Document signing, micropayments, transaction recording, stock trading, single sign-on, and more. … Is your organization going to resist this trend or take advantage of it?"

That sounds like hype, but as of February 2017, "blockchain" was the second-most-searched-for term on Gartner.com, with the search volume up 400% over the previous year. And that's on top of a 600% increase between 2015 and 2016. Interest, the consultancy said, is growing exponentially.

Blockchain finding its way into a variety of industries in 2018, said Cybric's Kail.

The number of relevant use cases for blockchain is rapidly increasing, as evidenced by the number of startups and investments in this area, Kail said.

"In addition to the obvious financial transaction areas, we will see companies leveraging distributed ledger technology for things such as digital identity along with authentication and authorization."
—Mike Kail

What concerns you about the coming year?

Looking ahead, IT Ops pros should expect to face a more complex technology environment that's fraught with a greater amount of risk—and fewer resources  to manage it all. As emerging technologies such as blockchain enter the mainstream, these trends will only become more pronounced.

"IT Ops assets are now both ephemeral and elastic, which presents clear challenges for IT leaders to measure what's important, provide what is needed, and overall stay out in front of requests from the various lines of business," said Kail. But the technology implementation is the easy part, he adds. "What will be challenging is driving the continual cultural evolution needed to truly power digital transformation."

This is just a snapshot of some of the more critical issues IT operations will face in 2018. What did we miss that didn't make the list? If you and your team have concerns that aren't mentioned here, keep the conversation going in the comments section below.

The European Union's General Data Protection Regulation (GDPR)—a document whose 88 pages contain nearly a hundred articles—passed in April of last year and goes into effect in May. GDPR applies to any firm—including US companies—that have personally identifiable information on European customers. 

Under the GDPR, citizens must be notified about any data collection in clear language and they have the right to both access the collected information and correct any errors. Data controllers must ensure that the data is used only for the stated purpose and protect both the data and its confidentiality.

Developers are key to complying with these regulations. Applications that collect and store information must be rewritten—and in many cases, redesigned—for more strict privacy.

Here are three ways that companies should work with developers to make sure they are ready for the GDPR.

1. The privacy buck has to stop somewhere

Privacy cannot just live in a written privacy statement, said Trevor Hughes, CEO and president of the International Association of Privacy Professionals.

"It is not about policies; it is about the way you build products and services inside your organization."
—Trevor Hughes

The GDPR requires that companies address a wide variety of privacy regulations. Larger companies—or those whose core activities focus on data—need a data protection officer (DPO), but more than half of companies aren't planning on hiring one or plan to wait until the second half of the year, after the GDPR takes effect, according to a survey by security firm Imperva.

Delaying the hiring of a DPO is not a showstopper, said Hughes, but every organization needs someone to lead the push for privacy, especially to push development efforts in the right direction. "Not every company has the size or the need for a chief privacy officer, but no matter what, there needs to be privacy leadership incorporated somewhere," he said.

Right now, however, many companies do not know if the GDPR applies to them. More than one third of businesses (37%) did not know if they needed to comply, while 35% believed that the regulations did not apply to their companies, according to a recent survey by WatchGuard Technologies. Among those who thought they did not need to comply with the EU regulations, one in seven collected information on Europeans deemed identifiable under GDPR.

People are cutting it a little bit close, said Tracy Hillstrom, director of product marketing at WatchGuard.

"If they are not yet understanding that they need to be compliant and they do, and they haven't kicked off a project today, then they could be in trouble."
—Tracy Hillstrom

2. Review data collection with developers, marketers

Developers and marketing professionals must understand that they may not be able to gather and use the same information as in the past.  Your executive leader in charge of privacy should meet with the engineering and marketing leads to set rules for the type of data to collect and how it can be used.

"That has to come from the director of engineering, or sometimes from the CISO or the data protection officer," said Tim Matthews, vice president of marketing for Imperva. "And I can imagine that there is going to be some very fraught conversations because you will have to balance having more users for your app or offsetting the risk for your company."

Training is an important part of getting the message out, since you don't want developers don't do something that puts the company at risk, said the IAPP's Hughes.

"A great example here are the engineers at Uber who created a function called 'God View,'" he said, referring to an administrative feature in Uber's service that allowed some internal employees to monitor rides in real time.

"Out of the gate, designers creating something called 'God View' should set off every alarm bell inside your organization."
—Trevor Hughes

3. Technology will likely be necessary

The GDPR requires that you design privacy into products and services and make privacy the default. For that reason, privacy impact assessments, which gauge the privacy implications of certain features in an application, should be completed regularly. But that's challenging because documenting privacy impacts can quickly become unwieldy.

"Any product or service that touches data—any upgrade, amendment, or new feature that touches data—that is, just about everything an organization is doing—needs to go through a privacy impact assessment," Hughes said. "A large organization may go through hundreds or thousands of impact assessments, and that means they need technology."

Most companies, especially larger ones, will technology that helps track compliance and turns requirements into design goals for developers. In the end, companies must determine whether or not they fall under the GDPR, and then strive for compliance. Developers are key to making it work.

"You are required to understand what data you have and demonstrate that you are complying with the GDPR. That has many organizations scrambling."
—Trevor Hughes

Get real about GDPR

The GDPR underscores that companies need to prioritize the protection of user data, and it highlights the importance of developers to that mission. Penalties are steep for noncompliance: Penalties can run up to 20 million Euros, or 4% of revenue, whichever is greater.

According to WatchGuard's survey, only 10% of companies thought they were ready to meet the deadline next May, while 44% believe that they need to comply but did not know how close they were to completion.

"The surprising part is how many people outside the EU don't realize that this is important to them," Hughes said. Even companies that believe they have data belonging to EU citizens don't always understand that the GDPR pertains to them.

Many large companies concerned with security and privacy have incorporated the issues into their development process, but smaller companies and their developers need to focus on privacy as well.

The way companies handle data and feed their marketing efforts also needs to change, said Matthews. "They are going to be told by someone that they will not be able to use all these great tools they have come to rely on. They may not be able to use those cookies, or they may not be able to use those mobile numbers."

"Some of the cool features that they are conceiving may not be allowed."
—Tim Matthews

Here are the six levels of testing so you can avoid getting caught flat-footed in the AI testing revolution.

Level 0: No autonomy

Congratulations! You write code that tests the application, and you're happy because you can run the same tests again and again on every release. This is perfect, because now you can concentrate on the most important aspect of testing: thinking.

But nobody's helping you write that automation code. And writing the code itself is repetitive. Adding any field to a form means adding a test. Adding any form to a page means adding a test that checks all the fields. And adding any page means checking all the components and forms in that page.

And the more tests you have, the more they will fail when developers make a sweeping change to the application. So check every failed test to verify whether you have a real bug or just a new baseline.

Level 1: Drive assistance

The autonomous vehicle serves as a good metaphor for this level of testing. The better the vision system an autonomous car has, the more autonomous it is. Similarly, the better an AI system can see your app, the more autonomous your testing will be.

AI should be able to see not only a snapshot of the Document Object Model (DOM) of the page, but a visual picture of it as well. The DOM is only half the picture—the fact that there is an element with text on it does not mean that the user can see it. Other elements may obscure the page, or the page might not be in the correct position. Visuals let you concentrate on the data the user sees.

Once your testing framework can see the page and look at it holistically, whether through the DOM or a screenshot, it can help write checks that you would otherwise write manually.

If you take a screenshot of the page, you can check all form fields and texts in it in one fell swoop. Instead of writing code that checks each field, you test all of them at once against the baseline of your previous test.

techbeacon_image_1.png

Figure 1: Checking the whole form in one fell swoop.

For this level of testing to work your testing tools need AI algorithms that can determine which changes are not really changes at all, and which are real.

AI technology today can assist you in writing test code by writing your checks. You're  still driving the tests, but AI can do some checks automatically.

Also, AI can check that a test passes. But when it fails, AI still must notify you so you can to check whether the failure is real, or happened because of a correct change in the software. You must either confirm that the change is good or reject it because it's a bug.

And having AI "see" the application means it checks the visual aspects of the application against a baseline—something that until now could only be done through manual testing. But you still need to verify every change.

Level 2: Partial automation

With Level 1 autonomy, the tester can avoid the tedious aspect of writing checks for all fields on a page by having AI test against a baseline. And you can have AI can test the visual aspects of the page as well.

But checking every test failure to verify whether it's a "good" failure or a bug can be tedious, especially if one change is reflected in many test failures. At Level 2, your AI needs to understand the difference in terms that the user of the application should be able to understand. Thus, a Level 2 AI should be able to group changes from lots of pages, since it will be able to understand that, semantically, those are the same change.

A Level 2 AI can group these changes, tell the human when the changes are the same, and ask whether to confirm or reject all the changes as a group.

techbeacon_image_2.png

Figure 2: Grouping similar changes with AI-powered visual test automation.

In sum, Level 2 AI helps you check changes against baseline and turns what was a tedious effort into a simple one.

Level 3: Conditional automation

In Level 2, a human still must vet any failure or change detected in the software.

A Level 2 AI can analyze the change but can't determine whether a page is correct or not just by looking at it. It needs a baseline against which to compare. But a Level 3 AI can do that and more by applying machine-learning techniques to the page.

For example, a Level 3 AI can examine the visual aspects of a page and decide if the design is off based on standard rules of design, including alignment, whitespace use, color and font usage, and layout.

techbeacon_image_3.png

Figure 3: A Level 3 AI will autonomously determine that this is a bug.

And what about the data aspects? A Level 3 AI can check the data and determine that all numbers up to now, in this field or that, must be in a specific range; that this field is an email; and that another must be the sum of the fields above it. It knows that in a specific page, the table must be sorted by a given column.

Your AI can now evaluate pages without human intervention, just by understanding your design and data rules. And even if there were a change in the page, AI can understand that the page is still fine and doesn't need to be passed to a human for review.

AI looks at hundreds of test results and can see how things change over time. And by applying machine-learning techniques the AI system can detect anomalies in changes and submit only those to a human for verification.

Level 4: High automation

Until now AI has only run checks automatically. Humans still drove the test and clicked on the links (using automation software). Level 4 is where AI will drives the test itself.

Because Level 4 AI can examine a page and understand it just as a human would, it understands when it's looking at a login page versus a profile, registration or shopping cart page. And because it understands the page semantically, as a page that is part of the flow of interaction, AI can drive the tests. 

While pages such as login and registration are standard, most others aren't. But a Level 4 AI will be able to look at user interactions over time, visualizing the interactions, and understand the pages and the flow through them, even if they are pages of a type the AI system never saw.

Once AI understands the type of page, using techniques such as reinforcement learning—a type of machine learning—it can start driving tests automatically. It can write the tests, not just the checks for them. 

Level 5: Full automation

This level is science fiction for now. At this level, the AI would be able to converse with the product manager, understand the application, and fully drive the tests by itself.

But given that no one has been able to understand a product manager's description of an application, AI-5 would need to be much smarter than humans.

The current state-of-the art

Advanced tools are currently at Level 1, and they're progressing nicely with Level 2 functionality. While tool vendors are working on Level 2, Level 3 AI still needs work.  But it's doable.

A Level 4 AI is far in the future, but in the next decade or so you can expect to see AI-assisted testing without the nasty side effects.

So don't start looking for another job because you think computers are going to automate all software testing, including visual tests. Software testing is similar to driving in some aspects, but it is more complicated because these systems must understand complex human interactions. AI today doesn't understand what it's doing. It merely automates tasks based on lots and lots of historical data.

Your role is to be the automator, not the automated. Test and QA teams have an opportunity to leverage new automation techniques that will spur new ways of working. Once you're freed up to focus more on how to test and ensure quality, as automation does its job, you'll be able to create new value for the business.

Want to learn more about this topic? Post your questions and comments below. Or, better yet, attend my presentation, "AI, Please Test My Apps," at the 2018 Automation Guild online conference starting today, January 8. Registered attendees can also watch presentations at any time after the event.

The DevOps Quadrant Maturity Model lets you do exactly that. Use it to assess where you are in your DevOps transformation as well as where your organization needs to go next. It's also a great tool for IT executives who want to get going on a DevOps transformation and want to develop a strategy for getting started.

Get real-world about your DevOps transformation

The DevOps Quadrant Maturity Model derives from the real-world transformations of Fortune 500 organizations in many different industries. To frame the space representing non-DevOps and DevOps organizations, the model establishes two axes. The x axis includes the phases of a software development lifecycle: define, plan, code, build, integrate, test, release, deploy, and operate. The model divides these phases into two groups based on traditional organizational and cultural boundaries and observation of industry practices: agile upstream and agile downstream. 

y-axis-dmm-crop.jpg

The y axis displays the levels of adoption for agile practices that you can deploy at the team, workgroup, or enterprise level.  

Which quadrant is your organization in?

Quadrant 1: Team-level agile (upstream)

Organizations in the first quadrant have one or more teams that have adopted agile development practices and implemented continuous integration (CI). But tools, project management, and development processes are team-specific, and that can cause disconnects at the enterprise level. These organizations also take a "water-Scrum-fall" approach.

Quadrant 2: Team-level CD (downstream)

Those in quadrant two have applications supported by automated testing, dynamically provisioned environments, configuration management, and containers. Team-level CI extends to continuous delivery (CD), and the integration of development tools with delivery tools. Organizations in quadrant two focus on standardizing practices and shared solutions for tools and infrastructure management, and on refining the overall development processes.

Quadrant 3: Enterprise agile (upstream)

IT teams in the third quadrant follow standard, enterprise-wide agile development practices. They have aligned the overall business with agile specifications, and have adopted common project management and development processes. Organizations in this quadrant should integrate tools at the enterprise level in order to move forward.

Quadrant 4: Enterprise DevOps (downstream)

These organizations are among the 10% that made it to this phase of DevOps adoption. They have standard tools and processes for automated test and deployment, dynamically provisioned environments, and use configuration management and containers. The overall business and its customers are aligned with agile specifications, and the organization has integrated its tools with a common platform to support reporting, security, governance, and traceability. These organizations foster a culture of collaboration, learning, and innovation. 

Barriers to enterprise DevOps

The benefits of enterprise DevOps are clear, so what’s standing in the way for the 90% of organizations that aren’t there yet? It comes down to the DevOps trinity: people and culture, process and practices, and tools and technology. When organizations attempt a DevOps transformation without considering all three, they often face unmet expectations. 

Consider an organization that dismisses cultural change as a requirement, but seeks to accelerate the delivery of higher-quality software using only tools and technology. It makes significant investments in automated tools for testing, but lacks a quality-first culture. Teams go through the motions, but don’t actively take steps to ensure quality. But if they emphasized cultural change, the organization would have set quality goals and implemented automation and collaboration across silos to correct problems early and quickly.

In contrast, an organization that's committed to cultural change but doesn’t  adopt agile methods and automated tools will find the cultural changes too difficult to sustain. It will suffer from too many manual steps, heavyweight processes, cumbersome legacy tools, and so on. The organization won’t achieve faster delivery, and the initiative will fail.

So focus on what processes you need to have in place, what technology you need to make those processes seamless, and what cultural changes you require to get your initiative off the ground in a sustainable manner.

The right way to get started with DevOps

When launching a DevOps initiative be pragmatic, not dogmatic. You can't know everything as you start, so approach and plan your initiative using the knowledge you have, and adjust along the way.

Patience is key to figuring out which processes are right for your teams. Treat the transformation as an evolution by pursuing a gradual progress that your team can sustain over time. If  your teams can't keep up, they'll make mistakes.

Choose your first project wisely. Pick one that's high-value, low-risk and assign it to a team that can demonstrate what DevOps can do. The right team will be innovative, constantly driving change in the product development process, willing to test new tools and processes, and open to change.

Finally, showcase the value of DevOps to upper management and other teams. Keep track of improvements, and document everything so you can show management that your initiative is successful and should be expanded. People will quickly see the benefits, and you’ll be one step closer to enterprise-wise DevOps. 

Whether you’re far along your DevOps journey or just getting started, the DevOps Quadrant Maturity Model is a great way to see where you are and map out where you need to go. And it’s not just about going through the motions and investing in the right tools, so be dedicated and patient with the process. The process isn't easy, but it's definitely worthwhile.

The Journey to Hybrid Cloud: A Design and Transformation Guide

GET REPORT

TechBeacon's top 10 quality stories for 2017 highlight the ongoing changes in the QA landscape. They touch upon the importance of API testing, propose that QA engineers need to learn coding, explain the benefits of using the cloud for functional testing, and offer tips on how to identify the best tools and approaches to help you do your job better. 

How to perform API testing with REST Assured

Automated testing of APIs has become a critical requirement for organizations because of the increasing role that APIs play in mobile, the Internet of Things, and other emerging application environments. Many tools are available to help you write automated tests at the API level. One of the most popular, says writer and independent consultant Bas Dijkstra, is the REST Assured Java library, which offers a domain-specific library language for creating powerful tests for RESTful APIs. Here's a look at how to use it in your testing.

Functional testing in the cloud: 7 keys to better software

The cloud can help improve the quality and velocity of functional testing practices. But how do you know whether cloud-based functional testing is the right approach for you? What are the factors you need to consider when comparing cloud versus non-cloud options for your testing requirements? Micro Focus' Antoine Aymer walks you through the seven things to keep in mind when looking at cloud options for functional testing.

The state of test automation tools: Top trends and challenges

As 2018 gets underway, expect to see some test automation tools converge, more artificial intelligence (AI) capabilities embedded in the tools you use, more innovation coming from commercial tools vendors, and a change in mindset as more organizations realize that maybe Selenium isn't the only thing you should have in your toolbox. Linda Rosencrance gives a full rundown of the state of software test automation tools.

How tech giants test software: There's no one way for QA

Organizations seeking to improve their quality and testing practices can learn from technology giants such as Amazon, Google, and Microsoft. The processes these companies put in place to ensure the quality of their products and services have played a big role in their success. Independent consultant Bas Dijkstra reviews the QA practices of the world's five best-known technology companies and explains why there's no one single recipe for getting it right.

World Quality Report 2017-18: The state of QA and testing

Software testing and QA practices have not kept up with the pace of change around development driven by DevOps and agile. The adoption of initiatives such as test data management and automation, in particular, continue to languish despite growing awareness of the need for these practices. Ericka Chickowski reports on the state of QA and testing practices based on the findings in Capgemini/Sogeti Group's World Quality Report 2017-18.

The No. 1 unit testing best practice: Stop doing it

Unit tests, which form the first layer of the testing pyramid, are something that all developers are trained to believe in and embrace. Unit tests may be are fast, but they're also a pain to maintain, especially when developers are encouraged to write as many as possible. HPE senior systems architect Vitaliy Pisarev explains why he gives preference to integration/systems, even though they're slower than unit tests.

Coding is key to a test automation career: Are you prepared?

QA engineers have traditionally been non-coders. But if you are a manual tester trying to figure out the ins and outs of testing automation, it helps to know how to code, says T.J. Maher, automation developer at Adventures in Automation. He explains why QA engineers who want to get into automated testing need to learn programming, and he offers five steps for gaining employment as an automation developer.

5 ways to drive your automation engineers away

Automation engineers are in high demand. The growing adoption of DevOps and agile practices has created the need for professionals with both testing and software development skills at many organizations. Companies can spend months finding a qualified automation engineer, so it is important to find ways of retaining them for as long as possible. Twitter senior software engineer Angie Jones highlights five mistakes organizations can make that will cause an automation engineer to quit.

5 top open-source API testing tools: How to choose

DevOps, continuous integration/continuous delivery (CI/CD), and agile practices have dramatically accelerated application delivery. To ensure that your applications are ready to ship in this environment, you need to make API testing a core component of your automation strategy. But choosing the right testing tools can be hard, given the wide range of choices. TestTalks founder Joe Colantonio highlights five of the best API testing tools out there, with helpful advice on choosing the one best suited to your requirements.

5 ways AI will change software testing

The 2016-17 World Quality Report predicts that AI tools could soon start helping QA teams overcome some of the growing challenges associated with software testing. As a tester, do you know how to use AI to verify and test code suites? How will testing requirements change as AI starts getting embedded in production applications? Beaufort Fairmont CEO Paul Merrill examines five ways that experts believe AI will change testing.

Here is our shortlist of the most popular agile and lean software development conferences in 2018. We've listed them all, although not all dates, locations, and pricing were available at publication time, especially for those events taking place later this year. In those cases, we have provided historical information on the event to give you an idea of what to expect, and what you'll get out of attending. And keep checking back: We'll update this guide as more information becomes available.

February

DeveloperWeek

Twitter: @DeveloperWeek / #DevWeek18
Web: developerweek.com
Date: February 3-7
Location: Oakland and San Francisco, California, USA
Cost: Prices range from $55 for keynotes and DevWeek Expo to $1,445 for access to all events; additional conference packages can be found here.

DeveloperWeek 2018 is the world’s largest developer expo and conference series. Last year's "amazing Dev Network event brought no fewer than 8,000 developers ... for one week of incredible demonstrations, displays, and conversations," wrote Ahsan Awan. The power of code is the theme: The more software and the cloud integrate with any industry, the more we will see innovation move from "manufacturing" to "coding."

Who should attend: Developers, development managers and executives, artificial intelligence experts, and DevOps pros

Agile Open Northwest

Twitter: @aonw / #aonw2018
Web: agileopennorthwest.org/
Date: February 5-7
Location: Seattle, Washington, USA
Cost: $300

This event, sponsored by a coalition of practitioners and advocates intent on advancing agile, fittingly organizes its approach to sessions using open-space self-organizing principles and technology (OST) so that everyone has a chance not only to learn but also to contribute. Attendees can expect to grow their knowledge of lean and agile principles. Anyone with expertise in a topic related to the conference theme can propose relevant sessions. "Those discussions are universally great," wrote Eric Gunnerson.

Who should attend: Experienced, collaborative, committed agile practitioners and learners, including developers, testers, and business people

Agile Open California

Twitter: @AgileOpen / #AgileOpenCA
Web: agileopencalifornia.com/san_diego.html
Date: February 15-16
Location: San Diego, California, USA
Cost: $225

This event, sponsored by a coalition of practitioners and advocates intent on advancing agile, fittingly organizes its approach to sessions using open-space, self-organizing principles and technology so that everyone has a chance not only to learn but also to contribute. Attendees can expect to grow their knowledge of lean and agile principles.

Who should attend: Experienced, collaborative, committed agile practitioners and learners, including developers, testers, and business people

April

Agilia Conference / Agile Management Congress

Twitter: @agiliaconf / #agilia18
Web: agilemanagementcongress.com/
Date: April 9-13
Location: Olomouc, Czech Republic
Cost: €490 ($328.66) conference only; additional conference packages can be found here.

The Agile Management Congress is a subtrack of the Agilia Conference showcases improving business performance with agile, studying the topic through a number of different lenses: culture, software development, agile contracting, innovation, and more. It will follow its tradition of verifying presentations for authenticity and of offering fresh content thanks to limiting speakers' appearance to once every two years.

Who should attend: Managers (technical, finance, HR, operations, etc.), owners of companies, and board members interested in agile methodologies

Lean Kanban North America

Twitter: @leankanbanNA / #LKNA18
Web: lkna18.leankanban.com
Date: April 9-13
Location: Seattle, Washington, USA
Cost: Ranges from $1,799 for two-day conference pass to $3,599 for conference and leadership retreat; additional conference and training packages can be found here.

Attendees of Lean Kanban can connect with coaches and practitioners who use the Kanban and lean methodologies. Interactive workshops and sessions offer insight into enterprise scaling, the Kanban maturity model, upstream Kanban, forecasting, product management, visualization, and much more.

Who should attend: New and mature Kanban practitioners, coaches, trainers, consultants, and thought leaders

North American Global Scrum Gathering Minneapolis 2018

Twitter: @ScrumAlliance / #SGMSP18
Web: scrumalliance.org/sgmsp
Date: April 16-18
Location: Minneapolis, Minnesota, USA
Cost: Ranges from $1,350 for members to $1,600 for non-members; additional conference packages can be found here.

The event touts an innovative and actionable experience for Scrum practitioners. There are nine education themes on Scrum and agile topics, including government and regulated industries and organizational transformation, one-on-one coaching, and the chance to gain credit toward a Certified Scrum Professional® certification. "There was a lot of variety and a lot of great content," wrote Christopher Trudeau of his past experience.

Who should attend: Scrum and agile practitioners, agile enthusiasts, and business leaders and managers

Agile-Lean Ireland

Twitter: @AgileLeanIrl / #ALI2018
Web: agileleanireland.org/
Date: April 26
Location: Dublin, Ireland
Cost: €100, or €120 for late registration (after March 25)

This community-led conference mixes open-space sessions with traditional conference workshops, presentations, and lightning talks. The conference includes speakers from high-profile companies such as Spotify and Lego. Authors and practitioners of agile and lean methodologies are prevalent at this conference.

Who should attend: Agile/lean coaches, Scrum masters, product owners, leaders, and developers

deliver: Agile 2018

Twitter: @agilealliance / #deliveragile2018
Web: agilealliance.org/deliver-agile-2018
Date: April 30-May 2
Location: Austin, Texas, USA
Cost: Ranges from $1,195 for early-bird members to $1,395 for non-members; group rates available.

The uniting of engineering and architectural topics under the umbrella of agile thinking forms the core of deliver: Agile 2018. It's a conference targeted at the multiple disciplines involved in agile ecosystems. Whether you're looking to support your agile practices with improved collaboration skills or better user experience implementations, its sessions will provide guidance from real teams who have been there.

Who should attend: Developers, system administrators, QA professionals, UX designers, infrastructure engineers, data scientists, cloud specialists, and DevOps and other agile discipline practitioners

May

Agile and Beyond

Twitter: @AgileAndBeyond / #AAB18
Web: agileandbeyond.com/2018/
Date: May 16-18
Location: Ypsilanti, Michigan, USA
Cost: $259-$279; group rates available

The all-volunteer Agile and Beyond conference will feature about 100 breakout and workshop sessions over the course of two days, focusing on agile principles and practices—as well as covering topics that it says will help make people and companies "awesome." The conference supports both novices and experts.

Who should attend: Designers, developers, and executives interested in lean and agile

Mile High Agile

Twitter: @agiledenver / #milehighagile
Web: milehighagile.org/
Date: May 21-22
Location: Denver, Colorado, USA
Cost: N/A; last year's tickets cost $525.

Agile Denver, the city's nonprofit community for agile and lean methods, sponsors this conference to help agile novices, intermediates, and experts grow, adapt, and evolve. Alan McKellar wrote that he enjoyed inspiring conversations at the conference, saying, "the community recognizes the need to look beyond practices and process to evolve and continue to grow as craftsmen."

Who should attend: Agile practitioners, technical professionals, team leaders, managers, executives, and organizational change leaders

June

Agile Dev West

Twitter: @TechWell / #AgileDevWest
Web: adcwest.techwell.com
Date: June 3-8
Location: Las Vegas, Nevada, USA
Cost: Ranges from Super Early Bird pricing of $595 for the one-day Agile Leadership Summit to $3,995 regular pricing for the whole conference and three-day training class. Additional packages can be found here.

Here newbies have a chance to get up to speed quickly, and experienced pros can take their teams to the next level with hands-on, in-depth workshops. Another benefit is that the conference is held in conjunction with Better Software West and DevOps West, allowing you to choose from three distinct programs.

Who should attend: Agile teams and leadership, digital transformation teams, product owners, and managers in the areas of automation, development, test, and release

DevOps Enterprise Summit (DOES) London

Twitter: @DOES_EUR / #DOES18
Web: https://events.itrevolution.com/eur/
Date: June 25-26
Location: Intercontinental London - The O2, London, UK
Cost: N/A; last year's costs ranged from $1,250 to $2,150.

DOES is more than just a conference about technical and architectural DevOps practices. The event also covers the methods needed to lead widespread change efforts in large organizations. The goal is to give leaders the tools and practices they need to develop and deploy software faster. Attendees can expect to see speakers from many large companies and also engage in ad hoc discussions within a great community and learning environment.

Who should attend: Leaders of large, complex organizations implementing DevOps or agile principles and practices, developers, IT operations specialists, CxOs, software architects, and systems and network admins

August

Agile Alliance Agile2018

Twitter: @AgileAlliance / #Agile2018
Web: www.agilealliance.org/agile2018
Date: August 6-10
Location: San Diego, CA, USA
Cost: Agile Alliance members: $1,649-$1,999; non-members: $2,399; academics: $999 (Unconfirmed, based on 2017 prices)

Agile2018 will mark the 17th year that Agile Alliance has hosted this international conference. Being the world's foremost agile event, the conference sells out quickly, so early reservations are a must. The conference features the most distinguished speakers in the field of agile software development and includes over 200 sessions.

Who should attend: Software project managers, product owners, developers, testers, and basically anyone with a significant interest in agile practices

September

Atlassian Summit

Twitter: @Atlassian / #atlassiansummit
Web: atlassian.com/company/events/summit-europe
Date: September 4-6
Location: Barcelona, Spain
Cost: N/A; last year's costs ranged from $799 to $1,299.

Development and collaboration software company Atlassian will provide developers in its user community the opportunity to enhance their skills and give teams the chance to compete to create a new capability that works with its Atlassian Tool Suite during the always-anticipated ShipIt Live hackathon. Its take on the user conference includes expanding beyond product talk to exploring issues such as how to create cultures of innovation and diversity in the tech industry.

Who should attend: Members of the Atlassian user and partner community

Agile Open Southern California

Twitter: @AgileOpen / #AgileOpen
Web: agileopencalifornia.com/southern_ca.html
Date: September 7-8
Location: Irvine, California, USA
Cost: N/A; last year's cost: $225

Like the Agile Open California event in San Diego, this event is hosted by a coalition of agile practitioners and advocates. The event uses open-space principles, helping all participants learn about and contribute to the discussion.

Who should attend: Experienced, collaborative, committed agile practitioners and learners, including developers, testers, and business people

Agile Cambridge

Twitter: @acconf / #agilecam
Web: agilecambridge.net
Date: September 26-28
Location: Cambridge, UK
Costs: £588 (including VAT) for Super Early Bird full-conference ticket (through March 15)

Agile Cambridge is described by its organizers as "a practical, hands-on agile software development conference" with a goal of letting participants interact with and learn from each other and from industry leaders. The event is focused on agile and lean experience sharing and learning by doing.

Who should attend: Agile practitioners looking for a small conference focused on hands-on training and learning

October

Agile Open Northern California

Twitter: @AgileOpen / #AgileOpen
Web: agileopencalifornia.com/northern_ca.html
Date: October 3-4
Location: Berkeley, California, USA
Cost: N/A; last year's cost: $250

The sister event to Agile Open Southern California and Agile Open California, it shares their approach to sessions using open-space principles, helping all participants learn about and contribute to the discussion.

Who should attend: Experienced, collaborative, committed agile practitioners and learners, including developers, testers, and business people

ASAS

Twitter: @Avisi_ASAS #ASAS 2018
Web: asas.nl/
Date: N/A; previous conferences held in October (2017) and September (2016)
Location: N/A; previous conferences held in Arnhem, Netherlands
Cost: €250 to €550

This conference emphasizes helping companies improve software development with new patterns, best practices, and agile principles, in order to deliver the right solutions for their users. Bart Hamer, application architect at CCV, stated that he attends "to learn about the latest software architecture ... trends and to get some wake up calls (usually provided by the ASAS keynote speakers)."

Who should attend: Technical leads, software architects, business analysts, and engineering directors

November

Agile Dev East

Twitter: @TechWell / #AgileDevEast
Web: adceast.techwell.com
Date: November 4-9
Location: Orlando, Florida, USA
Cost: Ranges from $595 for one-day Agile Leadership Summit to $3,995 for the full conference. Training and certification packages range from $3,495 to $3,995; conference packages range from $1,595 to $3,995; additional conference and training packages can be found here.

At Agile Dev East, newbies have a chance to get up to speed quickly, and experienced pros can take their teams to the next level with hands-on, in-depth workshops. The conference is held in conjunction with Better Software East and DevOps East, allowing you to choose from three distinct programs.

Who should attend: Agile teams and leadership, digital transformation teams, automation, developers, testers, release staff, product owners, and managers

DevOps Enterprise Summit (DOES) San Francisco

Twitter: @DOES_USA / #DOES18
Web: events.itrevolution.com/us
Date: November 13-15
Location: San Francisco, California, USA
Cost: N/A; last year's costs ranged from $1,250 to $2,150.

DOES is more than just a conference about technical and architectural DevOps practices. The event also covers the methods needed to lead widespread change efforts in large organizations. The goal is to give leaders the tools and practices they need to develop and deploy software faster. Attendees can expect to see speakers from many large companies and also engage in ad hoc discussions within a great community and learning environment.

​​​​​​Who should attend: Leaders of large, complex organizations implementing DevOps or agile principles and practices, developers, IT operations specialists, CxOs, software architects, systems and network admins

That concludes our list! For a more dense list of Agile Alliance-related workshops and events, check out the Agile Alliance event calendar too.

Review the options and make your choices soon: Prices may vary based on how early you register. Also, remember that hotel and travel costs are generally separate from the conference pricing.

What are your favorite conferences and why? Post your comments below, and let us know if there are any other events or conferences we missed.

Image: Courtesy of LeanKanban Week

Other top security stories for 2017 covered everything from application security to information security/cybersecurity. TechBeacon security stories that hit a nerve with readers spanned a wide range of topics, including best practices for microservices security, ways to combat fileless malware, threat modeling, the security impact of serverless computing models, and the importance of choosing the right metrics to measure security effectiveness.

Here's our top 10 list of stories that defined the state of security in 2017.

8 best practices for microservices app sec

No software architecture is entirely free from security considerations, including microservices. While a few microservices features help bolster security, others  accentuate security problems. For example access control is problematic in monolithic environments, and even more so in a microservices setting. ServisBOT senior software engineer Marco Troisi highlights eight best practices for securing your microservices apps.

5 application security metrics that should matter to your team

If you want to improve application security you need to know how well your current practices are working. You must measure progress to know if you are making any. How many of your applications are covered by secured development practices? How long does it take for you to address vulnerabilities? Do you have a handle on your flaw-creation rates? Independent technology journalist Robert Lemos reviews the five most important metrics for application security.

How your security team can combat new fileless malware

Fileless malware can create big headaches for security teams because it's so hard to detect. Unlike conventional malware tools, fileless malware resides entirely in memory and runs its payload there. Often this malware is designed to erase all traces of how it was delivered to a system, making it all but invisible to security tools. John P. Mello has the lowdown on measures that enterprises can take to mitigate the threat posed by this new breed of malware.

Why OWASP's Threat Dragon will change the game on threat modeling

Identifying and eliminating potential security vulnerabilities in a business process or design before you begin writing code for a new app is a great way to minimize vulnerabilities in software. Any enterprise that embeds such threat modeling within its development process is laying the foundation for creating more secure software, but it needs the right tools for the job. Security Journey CEO Chris Romeo explains why OWASP's Threat Dragon is the best tool for driving enterprise adoption of threat modeling.

Securing serverless apps: What IT Ops needs to know

Serverless apps are not a new concept, but adoption of the model is growing among enterprises. Among those pushing greater use of serverless apps are major cloud vendors Amazon, Microsoft, and Google. Trend Micro vice president Mark Nunnikhoven explains what you need to know about the security implications of serverless applications. He reviews the most secure designs, how to implement security, and what IT Ops need to understand about the trend.

Security liability is coming for software: Is your engineering team ready?

The era when software developers could get away relatively unscathed with serious security vulnerabilities in their products is quickly coming to an end. With a constantly increasing range of products and services being digitized, the consequences of failure associated with security vulnerabilities has become greater, and in some cases could potentially cause death or bodily harm. Robert Lemos explains why standard EULAs will soon no longer be enough to stave off liability for failures arising from software vulnerabilities.

How to keep your container secrets secure

Passwords, API keys, and access tokens help keep your source code secure, so it's vital to ensure that they do not fall into the wrong hands. But do you know how to keep these secrets safe in a container environment? What are the measures you need to take to secure keys, passwords, and tokens and ensure that only people with the right to access your source code can do so? Learn about the four actions you can take today from Liz Rice, technology evangelist at Aqua Security.

How to build the best cyber-threat hunting team

A growing number of organizations have begun using threat-hunting practices to proactively investigate and chase down security threats on their networks. However, there are few overarching standards or processes around threat hunting, and many organizations do it only when necessary rather, than on an ongoing basis. Robert Lemos examines current approaches to threat hunting and discovers four tips on how to do it right.

The 3 most crucial security behaviors in DevSecOps

Modifying security behaviors can help transform your DevOps team into an army of security practitioners. By instilling in them the importance of following threat modeling, red teaming, and code review practices, the security team can get DevOps to embrace security more readily, says Security Journey's Chris Romeo. He drills down into the three behaviors, and explains how they can help transform DevOps security.

Beyond two-factor: How to use U2F to improve app security

Just because two-factor authentication is better than using only a username and password doesn't mean it is infallible. 2FA has its own set of problems, which hackers have in recent times exploited successfully. Johanna Curiel, co-founder of Ossecsoft, takes a look at the open-source Universal 2nd Factor (U2FA) technology, reviews what it is, and explains why it offers application developers a better alternative to regular 2FA.

It's important to read between the lines of the product announcements. The practical insights below should help you navigate the rapidly evolving landscape of cloud computing.

There are seven key trends in cloud computing wrought by Amazon Web Services' (AWS) Lambda, which Kubernetes is beginning to leverage and which your company should at least pay attention to, if not incorporate into its own roadmap. (AWS's Lambda lets you run code without having to provision or manage servers.)

1. The cloud is evolving from operating system aggregator to the new OS

In the early years of AWS, Amazon spent most of its AWS R&D cycles turning hardware procurement and management into a software problem; the introductions of Amazon's S3 object storage and Elastic Compute Cloud (EC2) are the clearest examples of that. As a foundation for its infrastructure-as-a-service (IaaS) offering, AWS built out a cloud equivalent of the traditional hardware abstraction layer (HAL).  

Having completed that, AWS progressed up the stack, focusing next on building out a complement of quality-of-service (QoS) components and associated APIs, including queuing and notification systems, a system registry in DynamoDB, a scheduling subsystem, system-wide logging, and more.  

Finally, in 2015 AWS Lambda reached the general-availability status. Amazon filled in the final piece of the world’s first distributed cloud OS, complete with a HAL, QoS components, a software development kit, and a runtime. And while an operating system textbook from the 1990s might not characterize AWS as an OS, or EC2 and S3 as part of a HAL, the definitions have changed.  

What makes AWS Lambda (and its counterparts on other cloud platforms) so significant is that it has transformed how people think about the cloud's value proposition and whom that value proposition targets. In positioning itself to capitalize on big data, the Internet of Things (IoT), machine learning, and other changes rippling through the industry, Amazon recognized the need to graduate its wares from platform-as-a-service (PaaS) to toolchain-as-a-service (TCaaS) offerings. 

The distributed cloud operating system

Figure 1: The distributed cloud operating system, function-as-a-service (FaaS) and toolchain-as-a-service (TCaaS) platform. Source for all figures: Dean Hallman, All Things Open Talk, 2016.  

PaaS offerings such as Amazon's Elastic Beanstalk are tightly coupled, vertically integrated software stacks provided as turnkey foundations on which to build applications. TCaaS offerings are loosely coupled, horizontally integrated cloud applications that codify how and when various independent software systems should collaborate to solve some business need. (Integration platform as a service, or IPaaS, is related to TCaaS, but the latter is generally lower-level and more developer-oriented.) AWS Lambda and the events that trigger it arose as a means of building the glue logic to lace big data, the IoT, and future toolchains together (see Figure 1).

With the arrival of cloud-based toolchains and the requisite infrastructure to support them, AWS is evolving from an aggregator of sub-operating systems to a provider of the OS itself. Instead of building and deploying applications in the cloud, you can now build and deploy applications on the cloud. Other cloud vendors have since introduced similar function-as-a-service (FaaS) offerings, thereby extending this trend across a range of cloud providers.

The implications of this shift in value proposition and target audience aren't just tactical; they're strategic. Initially, most companies did not recognize the strategic implications, and they have continued to work with the cloud, both structurally and procedurally, in traditional ways.

2. Silos are breaking down, and devs are retooling

With the arrival of the cloud as OS, the siloed DevOps model (explained below) that was prevalent up until 2015 is now outmoded. In the traditional model, companies organize themselves into engineering, infrastructure, and QA departments, and the infrastructure team is responsible for dealing with most things related to AWS, Microsoft's Azure, Google Compute Cloud, and so on.

For example, infrastructure teams are usually responsible for building out the provisioning needed for Amazon's Virtual Private Cloud, EC2 instances, Amazon's Elastic Block Store volumes, and similar resource requirements.

Once provisioned, those resources pass over to the engineering team, which uses them as deployment targets. Engineering has historically been less knowledgeable about, and less concerned with, AWS's product portfolio, since AWS's products historically comprised components that turned systems administration functions into software problems.

DevOps was represented, at least in part, as the software that a system administrator wrote to provision the resources needed to meet engineering's requirements. Administrators often used infrastructure-as-code solutions such as Amazon's CloudFormation, Red Hat's Ansible, or HashiCorp's Terraform.

Prior to AWS Lambda, the relatively clean divisions between engineering and infrastructure, and the provision-and-pass model that governed their interaction, remained largely intact and defensible. But that all changed in 2015, when the cloud redefined itself as an OS.

The siloed DevOps model

Ironically, it was the siloed DevOps model (explained below), which was entrenched in the enterprise in 2015, that contributed to the slow uptake most enterprises experienced in recognizing the significance of AWS Lambda. That was because many professionals who were plugged into AWS product announcements and portfolio expansions were primarily concerned with infrastructure-related value-add. 

But here, for the first time, AWS Lambda was targeting an audience primarily of developers, who were generally less attuned to Amazon's ever-expanding product family. This presented a dilemma for teams using the siloed DevOps model. 

Here's what I mean by "siloed DevOps."

On the one hand, you had the infrastructure team, with access to many pieces of core technology, tasked with both the provisioning of production resources and guarding those resources against service disruptions over time.

On the other hand, you had developers who could not fully embrace and understand a new OS without having unfettered access to explore its capabilities. Imagine building a Mac or Windows app without knowing that the mouse event dispatch loop was a built-in core feature; you'd likely spend time building a less ideal solution.

In other words, developers building software for an OS that they were not allowed or inclined to explore fully would spin their wheels reinventing core features of the OS they didn't know were available out of the box.

To remedy this, developers should stop regarding AWS and other cloud platforms as the domain of the infrastructure team, and employers should stop restricting developers from having the access they require to educate themselves.

The cloud as OS yields a new class of solutions

Once developers stop regarding the cloud as infrastructure and start regarding it as an OS with capabilities relevant to their jobs, a new class of solutions presents itself.

For example, in 2015 I was working on a big-data platform when a new requirement arose. The clients could receive new or corrected data that arrived late—days or weeks after the fact. To fix this, they wanted to replace their manual processes with an automatic rewind that would recognize late arrivals, ingest the data, and recalculate any reports that were affected.

Initially, the team was considering traditional solutions to this problem, such as provisioning an EC2 instance running a REST API, allowing a sender to register late-arriving data, and scheduling the requisite processing. But upon learning of AWS Lambda, we decided to write a single Lambda that was listening for S3 write events within our data onboarding keyspace.

The cloud was expanding the set of solutions I could contemplate as a developer. The cloud was expanding its role from being the easel to becoming also the paintbrush and the color palette. This was the moment when I first recognized that the siloed DevOps model was breaking down and that AWS Lambda was forcing another expansion in the meaning of the term DevOps.

3. DevOps is being redefined—again 

The answer to the siloed DevOps dilemma is still taking shape, but some themes are starting to emerge.

Safer exploration inside a 'blast radius'

Principal among the emerging themes is the recent concept of "blast-radius containment." In other words, give developers a sandbox where they can learn, experiment, and blow stuff up while ensuring that they can't accidentally take out production in the process.

During AWS re:Invent 2016, Amazon outlined a strategy for limiting blast radius. In addition, some serverless tools and libraries provide minimal support for blast-radius confinement through sub-accounts and/or AWS's Identity and Access Management permission boundaries. 

Unfortunately, multiple account solutions are sometimes difficult to justify. Many managers are reticent to take on another accounts payable just so developers can have a sandbox. 

I've seen engineering managers put engineering team AWS accounts on their personal credit cards because the business was unwilling to open additional accounts. Or developers slapping down their own plastic and incurring small monthly charges with cloud providers because they understand that they need to know the cloud as an OS, even if their manager or employer does not see that.

Layering of dev and ops responsibilities

Another emerging theme in DevOps is to restructure the division of responsibilities between infrastructure and engineering as a more layered, or hierarchical, approach.

In this model, guarded environments such as production remain under the strict purview of the infrastructure team. But less guarded environments, such as dev and staging, have more relaxed permissions, affording fuller control to engineering teams. Conceptually, this is rather obvious—the real challenges lie in extending the traditional provision and pass methods to be more collaborative, hierarchical, and blast-radius-confining.

Trickledowndevops.png

Figure 2: Trickle-down DevOps (a.k.a. collaborative DevOps). 

In my talk at the 2016 All Things Open conference, I characterized this change as "trickle-down DevOps" (see Figure 2). I've since renamed it "collaborative DevOps," since the earlier siloed DevOps model is not truly collaborative—at least not between the engineering and infrastructure teams, where provision-and-pass has been the norm.

In collaborative DevOps, infrastructure and engineering teams use a common toolchain, methods, and key/value stores. This helps both teams specify, provision, deploy, and wire up a spectrum of target environments with permission levels governing which teams, team members, and continuous integration (CI) environments have full access to each target environment.

Modernizing DevOps to be a more collaborative workflow between infrastructure, QA, and engineering groups within the enterprise is a large topic that's beyond the scope of this article.  (Cloudbox, an open-source project I started, aims to deliver on the promise of collaborative DevOps for serverless- and container orchestration-based applications.)

4. Best practices for app configuration and staging are evolving

The oft-cited 12-Factor App—a methodology for building SaaS apps—gets a lot of things right. But the recommendations on configuration are sub-optimal, especially in the context of serverless computing, where keeping application settings in environment variables reduces flexibility and reinforces the silos that can stifle serverless application development.

While better solutions have started to emerge that are more compatible with the requirements of serverless computing, progress is needed on this front. And since application configuration and staging are critical points of baton-passing among infrastructure, QA, and engineering teams, the solution will play a key role in the ongoing evolution of DevOps. 

The AWS CloudFormation and the AWS Serverless Application Model (SAM) provide some guidance here. CloudFormation's support for stacksets and cross-stack references hints at a better direction, but packaging these settings within AWS's CloudFormation infrastructure-as-code service makes these improvements less readily available outside the infrastructure silo.

In addition, the serverless framework has implemented a rich variable passing-and-expansion language in a recent version that represents a step forward in building a more collaborative application configuration methodology. 

The Cloudbox project also aims to address this need. A Cloudbox is a hierarchical key/value storage framework that is stage- and permission-bounded. It collects the stage-specific configuration key/value pairs from a range of sources, both infrastructure- and engineering-oriented, and drives those settings into your serverless or non-serverless application. 

CloudBoxConcept.png

Figure 3: The Cloudbox application staging and configuration model.

A Cloudbox supersedes the notion of a stage by allowing one Cloudbox to be linked to others for ingress and egress purposes. A stage becomes just the final Cloudbox in this linked arrangement.

In addition to the configuration advantages of this model, it also allows the longer-lived Cloudboxes, which are managing persistent storage endpoints, to be managed by infrastructure, while the more transient, stateless, or computational Cloudboxes remain primarily an engineering concern.

And as illustrated in Figure 4, these transient, stateless Cloudboxes are loosely connected to "storage" Cloudboxes through a layer of indirection. As a result, each Cloudbox can be easily upgraded, recycled, and hot-reconnected back to production Cloudboxes, with no disruption to customer-facing UIs.

CloudBoxChain.png

Figure 4: Cloudbox chaining and application staging.

5. The serverless frameworks space is maturing, with new serverless app architectures emerging

When Amazon first released AWS Lambda, the offering included only the core FaaS feature. DevOps concerns, including automated deployment, configuration, security, and code sharing, were left as an "exercise for the reader." So third-party, open-source frameworks sprang up to fill this need.

Now there are many frameworks in this category, each with its own merits, but the most popular are Chalice (based on Python) and the Serverless Framework. And while this category has been maturing rapidly, until recently most frameworks lacked the security, CI support, and DevOps compatibility to be viable for use within an enterprise.

Your own resources might have to fill the void, for now

Back in 2016, I was building a serverless application for a client that was using the Serverless Framework as a foundation. One major problem was that Version 0.5.6 of the framework recommended that you create an IAM user with full admin permissions to the entire AWS account.  

The framework authors hadn't yet isolated which specific IAM permissions were required by the framework. But clearly, giving full admin privileges to AWS Lambda and its deployment scripts would have been a non-starter—the infrastructure team would have laughed me out of the room.

Moreover, the infrastructure team was responsible for establishing and disseminating IAM permissions, but it couldn't do that in a serverless framework that it knew nothing about and that conflicted with its own DevOps toolchain.

I ended up using my personal AWS account and spent many hours isolating every permission and resource the framework used. In this way I was able to narrow the individual permissions and resources that the framework required, and I then contributed my findings back to the community.

But the problems only compounded when I tried to add CI support to the project, since the framework wasn't compatible with my client's Jenkins environment. Meanwhile, the infrastructure team insisted that if I just used Kubernetes, I could sidestep all these concerns, because they'd handle the permissions and continuous integration requirements in their silo. 

Framework limitations and options

In more recent versions, the Serverless Framework team has done a great job of addressing these deficiencies. But this experience illustrated for me that, not only did serverless frameworks have a lot of room to mature, but they also could be doing more to streamline and simplify engineering's integration and interaction with the infrastructure team and its work products.

The Cloudbox project grew out of my recognition that serverless computing architectures and their cloud-as-OS presentation were challenging the silos between engineering and infrastructure in new ways.

AWS announced its own entrant into the serverless framework category, the AWS Serverless Application Model (SAM), in late 2016. SAM is different from other frameworks in the category in that it expands the platform, rather than working around absent capabilities in AWS.

More specifically, AWS SAM extends CloudFormation to support describing and deploying serverless applications.

But AWS SAM doesn't obviate the need for third-party frameworks. All of the third-party serverless frameworks, such as Chalice and Apex, have the option to replace their custom deployment features with an AWS SAM implementation, because AWS SAM overlaps their feature sets in this area. Beyond deployment, however, their value-add remains mostly intact and unchallenged by AWS SAM.

For example, Chalice, the Python-based framework that makes it easy to build REST APIs using API Gateway and Lambda, is entirely complementary to AWS SAM. And while the Serverless Framework does overlap somewhat with AWS SAM, it takes serverless computing into cross-cloud and multi-cloud directions that are well beyond the scope and business motivations of AWS SAM.

Programming options

One final area of growth in the serverless framework category are the programming paradigms and reusable libraries used in building serverless functions. For example, given the function granularity of serverless applications, should they use an object-oriented approach, a functional programming approach, or some combination of the two? And what framework capabilities can I derive from or incorporate into my serverless functions at runtime?

This is another area where Cloudbox offers some answers. It introduces a new programming paradigm, a hybrid between functional and object-oriented programming that builds on the advances of Functional Reactive Programming frameworks such as RxJs, but simplifies and extends the programming model.

This hybrid combines functional and object-oriented programming in an entirely new way, mixing in aspect-oriented programming, dependency injection, functional promises, intelligent retry, and railway-oriented programming. The result is a single, unified methodology that is both elegant and simplifying. It's a new way to build software that maps particularly well to serverless, cloud computing, and collaborative DevOps use cases.

6. Vendor lock-in fears are resurfacing, but cloud-agnostic solutions are appearing

While the pros of serverless computing outweigh the cons, some developers and organizations are understandably concerned with the vendor lock-in implications of building serverless applications.

On one hand, serverless is based on an event publication/subscription model, a design pattern intended to reduce tight coupling. On the other hand, the serverless handlers triggered by those events typically make use of provider-specific APIs, HAL, and QoS components, which is where vendor lock-in can occur.

Given that all major cloud platforms have now introduced FaaS offerings with capabilities similar to AWS Lambda, it should be feasible for you to migrate your serverless applications to a different cloud OS, just as long as you avoid vendor lock-in with your initial provider.

This situation is reminiscent of the desktop OS battles of the '80s and '90s. Enterprise application builders at the time wanted to avoid both tying their products too tightly to Windows, Mac OS, or Unix and constraining applications to a least-common-feature set that would cripple functionality.

Back then, products such as Wind/U, from Bristol Technologies, and Mainsoft's MainWin emerged, offering a means of avoiding vendor lock-in without imposing feature limitations.

In a case of history repeating itself, several products have since emerged that are modern counterparts to Wind/U and MainWin. WalmartLabs' OneOps, for example, bills itself as "one design, any cloud." OneOps is a comprehensive solution, but it requires some commitment to its architecture to take full advantage.

There are software design patterns and libraries out there that can help you minimize and localize cloud dependency in order to achieve an architecture that you can migrate with substantially less effort. Today, the Serverless Framework is the library that comes closest to delivering on the goal of a less invasive, software-abstraction approach to cross-cloud and multi-cloud support.

Cloudbox is a more recent entrant in this category and may be used exclusively or in combination with other serverless frameworks. 

7. A hybrid model is appearing: Serverless container orchestration

A new model I call "serverless container orchestration" is emerging. It offers a new approach to achieving a cloud-agnostic solution. If there is one trend in cloud computing that rivals the significance of serverless, it is the rise of container orchestration solutions, with Kubernetes at the forefront. Currently, serverless products are usually cheaper and more immediately deliverable. But as the scale and throughput of your application increases, the ROI lines will eventually cross, and at some point, Kubernetes will become more cost-effective.

The problem is, how do you make the transition from serverless to container orchestration without starting over? Is a middle ground feasible—a kind of serverless/container orchestration hybrid?

Initially, these solutions appeared to be competitive and mutually exclusive. But it's not turning out that way. New projects, such as IronFunctions, are pointing the way toward a future where serverless becomes a spectrum of possibilities, with AWS Lambda at one end and Kubernetes at the other.

In fact, AWS Lambda and the configuration values that constrain it are becoming a kind of accidental standard that serverless container orchestration products can use to build a Lambda-like environment on any cloud.

History can lend context here. In both the desktop OS wars of the '80s and '90s and the browser wars of the '90s through today, the industry encountered the problem of unevenness. One browser or OS supported one feature set, while another supported an overlapping, but distinct, feature set.

To deploy an application across a range of platforms with uneven capabilities, people began using a methodology that the JavaScript community dubbed "polyfill." As with an uneven drywall, the target platforms had hills and valleys that needed to be filled in and smoothed to support a single-source application on top.

For example, in the case of Wind/U, some years ago Bristol had to custom-build certain print driver functionality that was absent from Unix to support features, such as a print preview, that are standard on Windows.

With its built-in load balancing, failover capabilities, and multi-cloud support, Kubernetes makes for an excellent polyfill. While Kubernetes is not truly serverless, its management, scaling, and recovery capabilities have the potential to limit ongoing maintenance long enough to offer a reasonable approximation.

Any ongoing maintenance that's necessary can be easily peeled off and centralized with infrastructure specialists who possess a good working knowledge of Kubernetes, but fewer application specifics.

For these reasons, a Kubernetes-based approach can level-up cloud platforms that lack certain QoS components, including the FaaS runtime itself. And projects such as IronFunctions, built on Containership, which in turn is built on Kubernetes, are an example of this polyfill approach recurring in a new context: the distributed, multi-cloud OS. 

Be prepared for both technical and cultural shifts

Driven largely by the rising tides of big data, the IoT, and machine learning, the last two years have brought about a refocusing of cloud computing. New architectures, SDKs, and runtime components have combined to form a new breed of distributed OS. Principle among these new architectures are serverless and container orchestration technologies such as AWS Lambda, Docker, and Kubernetes. Each is transformative individually, but they may prove revolutionary when combined.

Hybrid serverless container orchestration tools, such as IronFunctions, have the potential to reduce or eliminate vendor lock-in. They can also provide a more flexible architecture that allows redeployment as app traffic increases, from a single source to more cost-effective serverless runtimes.

But the impact isn't just technical. There's also a cultural shift prompted by the recent evolution of cloud computing, as engineering teams take a more hands-on approach with cloud SDKs and serverless runtimes.

Frameworks have sprung up over the last two years to codify best practices and design patterns for building serverless applications. But for the most part, they've done little to address the evolving requirements of DevOps or to facilitate a more dynamic and collaborative interaction between the engineering, infrastructure, and QA silos that contribute to the traditional DevOps culture.

Looking ahead, expect to see new frameworks and architectures in 2018 that attempt to directly address and facilitate a more collaborative DevOps workflow for building serverless- and container-based applications. In addition, expect serverless container orchestration solutions to become more pervasive in cloud computing over the next three years, as people seek no-compromise approaches that let them build once and run anywhere.

The stage is set for the next round of battles and advances in cloud computing, and serverless and container orchestration strategies will play a key role in deciding the winners and losers. 

TechBeacon's top 10 IT Ops stories for 2017 examine some of the biggest changes that IT operations management and staff are having to deal with today and offer practical advice on how to understand and manage those changes. Our stories cover the changes that are being driven by general DevOps practices, such as the growing capacity crunch, the changes resulting from serverless computing, and the operational impact of the increasing use of containers in production environments.

How IT Ops can avoid the DevOps capacity crunch

IT operations management teams are under growing stress from having to balance their organization's security and operational requirements with the near-constant deployment demands of DevOps teams. The trend is squeezing already overloaded IT Ops teams to the breaking point and undermining their businesses' ability to operate. Rundeck co-founder Damon Edwards explains how self-service operations can help IT Ops better handle both the planned and, increasingly, unplanned work that is common in DevOps environments.

Epic IT Ops fails: The 5 worst blunders of 2017

IT Ops failures were quite the spectacle in 2017. Ericka Chickowskireviews the mistakes and mishaps of others you can glean some valuable lessons—even if it is simply developing an awareness that the cost estimates for these issues are probably not overinflated. If anything, they may be coming in a bit low. Learn from the blunders of Amazon, Equifax, British Airways, GitLab and others.

The business case for IT4IT: Why you need it. How to build it.

The Open Group's IT4IT reference architecture is designed to give organizations a framework for identifying and optimally managing the IT activities that help them become more competitive in business. IT4IT provides a resource for IT transformation, service management, and tools interoperability, but a lack of understanding and management buy-in is holding back adoption, says Daniel Warfield senior enterprise architect at CC and C Solutions. He offers his perspective on what IT4IT really means and how it can help.

The state of ITSM: Top trends and challenges for 2018

One of the prominent trends this year has been the collision of DevOps with traditional ITSM methodologies. John P. Mello reports that companies which embrace these new ideas have already begun accelerating their investments in analytics, management, orchestration, and new organizational models for IT, and they can expect a big payoff down the road. 

With immutable infrastructure, your systems can rise from the dead

Immutable infrastructure, built from service components that perform a limited function, can substantially reduce potential points of failure and improve the ability for engineers to offer support and resolve issues. However, in order to implement it, you should be willing to scrap the ideas and processes associated with tools such as Chef, Puppet, and Salt. Mike Johnston, infrastructure engineer at Supergiant.io, offers the lowdown on how immutable infrastructure concepts can apply in Kubernetes cluster environments.

An essential guide to the serverless ecosystem

Serverless computing approaches promise to improve security and operational practices, speed time-to-market, and enable pay-as-you-go pricing. While you might understand the essentials of a serverless architecture, do you really know what a serverless ecosystem looks like or all the options you have with it? Starbucks consultant Rafal Gancarz explains the nuts and bolts of serverless architectures and reviews the tools and frameworks you can use to build systems with a serverless stack.

Serverless computing has landed: How IT Ops can adapt

Serverless computing architectures eliminate the need for always-on server systems sitting behind every application and frees developers from having to write for specific physical or virtual machines in the cloud. What is less understood about the model, however, is its impact on IT operations management, service management, data center orchestration, and cloud management strategies. David Linthicum, senior vice president of Cloud Technology Partners, highlights the impact of serverless computing on IT operations and offers tips on how IT Ops should adapt.

30 essential container technology tools and resources

Container technology has transformed life not just for developers but also for testers, analysts, operations teams, and IT. But in order to get full value from it, you need to have the right mix of products and services for building, running, and managing containers efficiently. Excelon Development's managing consultant, Mattew Heusser, offers the lowdown on 30 essential tools and resources—including container architecture, deployment, management, and security products—that every organization should have on its shortlist when deploying containers.

The top 5 container adoption and management challenges for IT Ops

More companies are investing in containers, and those that are already using the technology are spending more on growing its use across their organizations. Much of the growth has come from developers who have adopted containers because the technology has made it easier for them to build and distribute applications. But with containers beginning to enter production environments, IT operations management and staff face new questions and challenges. Freelance writer Jennifer Zaino explains what some of those concerns are and how to deal with them.

Containers demand new data center designs: 5 steps to survive

Data centers are not typically designed to support container technologies, so the growing use of containers in production has begun to raise several important questions for data center managers. Among them are questions such as what it would take to re-architect data center infrastructure for containers, what operational tools and processes can help, and how data centers can be orchestrated for container computing. Cloud Technology Partners' David Linthicum offers tips on how data centers can survive the transition to container-based computing.

Here is our shortlist of the most popular mobile and IoT conferences in 2018. We've listed them all, although not all dates, locations, and pricing were available at publication time—especially for those events taking place later in 2018. In those cases, we have provided historical information on the event to give you an idea of what to expect and what you'll get out of attending. Keep checking back: We'll update this guide as more information becomes available.

January

IoT Evolution Expo

Twitter: @IoTEvolution / #IoTEvolution
Web: iotevolutionexpo.com
Date: January 22-25
Location: Orlando, Florida, USA
Cost: Prices range from free for an Expo Plus Pass when purchased in advance ($100 at the expo) to $3,595 ($3,695 at the show) for the Diamond Group Plan, which grants admission to all proceedings for three of your company's employees

IoT Evolution Expo focuses on how IoT can drive business transformation in all industries through improved operational efficiencies, revenue opportunities, and problem solutions. It features track sessions, an exhibit floor, case studies, special events, networking opportunities, and other events.

Who should attend: Developers, IT executives, business executives, device manufacturers, transportation companies, supply chain and logistics professionals, sensor and embedded system experts, and systems integrators

CES

Twitter: @CES
Web: cesweb.org
Date: January 9-12
Location: Las Vegas, Nevada, USA
Cost: Prices range from $300 to $1,700. Early-bird discounts available

This legendary massive consumer electronics conference and expo covers a wide range of topics, including mobile- and IoT-related sessions on security, digital entertainment, e-commerce, gaming, robotics, storage, education technology, mobile apps, and networking professionals.

Who should attend: Anyone interested in the latest and greatest consumer electronics

February

DeveloperWeek

Twitter: @DeveloperWeek / #DevWeek18
Web: developerweek.com
Date: February 3-7
Location: Oakland, California, USA
Cost: Prices range from $35 for an Expo Pass to $1,999 for a DevExec World Pass

According to the conference organizers, DeveloperWeek 2018 is the world’s largest developer expo and conference series, with over 8,000 participants across the DeveloperWeek 2018 Conference & Expo, the DeveloperWeek Hackathon, the Official Hiring Mixer, and citywide partner events.

Who should attend: Software engineers, entrepreneurs, and venture capitalists

Mobile World Congress

Twitter: @GSMA / #MWC18
Web: mobileworldcongress.com
Date: February 26-March 1
Location: Barcelona, Spain
Cost: Prices range from an Exhibition Pass, at €799, to a Platinum Pass, at €4,999

Described as a gathering for the entire mobile industry, the MWC is owned by GSMA, an industry group made up of 800 mobile operators and 300 mobile ecosystem companies. It's a big conference: In 2017, the MWC in Barcelona drew more than 108,000 attendees from 208 countries, and more than 2,300 exhibitors. Many smartphone and mobile device launches happen here.

Who should attend: App developers, operators, equipment vendors, and financial, marketing, and entertainment industry professionals

Embedded World Exhibition and Conference

Twitter: @embedded_world / #ew18
Web: embedded-world.eu/home.html
Date: February 27-March 1
Location: Nuremberg, Germany
Cost: Conference blocks range from €345 to €850 (VAT included); classes are €410 for a half-day and €610 for a full day

The Embedded World Conference is designed for developers and designers of embedded systems. Organizers say it is Europe's biggest conference devoted to embedded systems development, and it addresses all major topics in this sector, featuring papers and classes, with a focus on concrete solutions.

Who should attend: Developers and designers of embedded systems

March

Industry of Things World USA 2018

Twitter: @IoTClan / #IoTClan
Web: industryofthingsworldusa.com
Date: March 7-9
Location: San Diego, California, USA
Cost: Pricing options are Industry Delegate Premium, $2,695; Industry Delegate, $2,295; and Solution Provider, $3,495. Group discounts are available

Organizers call this conference "the leading Industrial IoT event in the USA." This event focuses on the impact of the industrial IoT on current business models and production processes across all major industries. It includes topics such as automation, machine-to-machine communication, interoperability, analytics, and new business models. Part of the Industry of Things World global event series, this conference has become the meeting point for senior executives who want to learn more about the industrial Internet.

Who should attend: IoT specialists and strategists, IoT novices, cloud computing adopters, big data analytics experts, and anyone else involved in a business's digital transformation

SXSW (South by Southwest)

Twitter: @sxsw / #SXSW
Web: sxsw.com/schedule
Date: March 9-18
Location: Austin, Texas, USA
Cost: Prices range from $445 to $1,450

While music and film are key elements of SXSW, the event also has a strong technology component. Topics include startups, blockchain, health IT, artificial intelligence, IoT, virtual reality, augmented reality, smart cities, digital media, software design and development, open source, mobile design, and user experience.

Who should attend: Developers, founders, and embedded developers

OpenIoT Summit

Twitter: @EventsLF / #openiot
Web: events.linuxfoundation.org/events/openiot-summit
Date: March 12-14
Location: Portland, Oregon, USA
Cost: Prices range from $200 to $850, depending on when you register

Created by technologists for technologists, OpenIoT Summit delivers the technical knowledge attendees need to deliver smart, connected products that take advantage of rapidly evolving IoT technologies. It is the only IoT event focused on the development of open IoT products, according to organizers. Experts from the world’s leading companies and biggest open-source projects will present on what you need to know to bring smart, connected products and solutions to market.

Videos of past years' sessions are available online.

Who should attend: Firmware developers, software developers, application developers, and system architects

DroidCon Boston 2018

Twitter: @droidconbos / #DroidconBos
Web: droidcon-boston.com
Date: March 26-27
Location: Boston, Massachusetts, USA
Cost: Prices: $99-$249

DroidCon is a global conference series focused on the Android developer ecosystem. The Boston conference is a two-day, full-immersion tech event driven by local communities and influencers. You can find dozens of other DroidCon events around the world, so be sure to check and see if there's a conference coming up near to your home.

Who should attend: Android developers and engineers

IoT6 Exchange

Twitter: @nGage_IoT / #IoT6
Web: www.iot6exchange.com
Date: March 27-29
Location: Ponte Vedra Beach, Florida, USA
Cost: An invitation-only event, Contact organizers for information

IoT6 Exchange features an in-depth agenda with keynote sessions, end-user panels, breakout sessions, and case studies. Topics include the industrial IoT, the IoT, machine learning, artificial intelligence, robotics and smart engineering, and asset and fleet monitoring and tracking. Complimentary airfare, accommodations, meals, and registration costs are provided to qualified attendees in appreciation for time spent out of office. Attendees must inquire to be eligible to receive a hosted invitation.

Who should attend: Operations, logistics, and manufacturing executives and decision makers

April

RWDevCon 2018

Twitter: @RWDevCon / #rwdevcon
Web: rwdevcon.com
Date: April 5-7
Location: Alexandria, Virginia, USA
Cost: Prices range from $999 to $1,999

RWDevCon is an Apple iOS conference focused on high-quality programming tutorials. Located just outside Washington, DC, RWDevCon is put on by teams at raywenderlich.com and 360iDev. Content focuses on hands-on tutorials, where attendees can code along with the presenter. 

Videos of past years' tutorials can be purchased for $199.

Who should attend: iOS and MacOS developers

Internet of Things Applications Europe

Twitter: Not available
Web: idtechex.com/internet-of-things-europe/show/en/
Date: April 11-12
Location: Berlin, Germany
Cost: Ranges from €99 for an exhibition pass to packages as high as €2,995.

Internet of Things Europe is collocated with the Wearable Europe and Sensors Europe events, and a ticket to any conference grants attendees access to the others. This event addresses real business opportunities for the IoT, not hype. Business models, case studies, opportunities, and profitability are all covered. It features specific market verticals, in addition to emerging technologies.

Who should attend: IoT developers, IT pros, and CxOs

AppBuilders

Twitter: @appbuilders_ch / #appbuilders18
Web: appbuilders.ch
Date: April 16-17
Location: Lugano, Switzerland
Cost: Prices range from $299 to $399, depending on when purchased

AppBuilders is a mobile technology conference featuring sessions for both iOS and Android developers. Conference speakers have featured lead iOS and Android developers from high-profile companies, including Viacom and The New York Times.

Who should attend: iOS and Android developers

May

Microsoft Build

Twitter: @msdev / #msbuild
Web: build.microsoft.com
Date: May (unconfirmed)
Location: Seattle, Washington, USA (unconfirmed)
Cost: $2,195 in 2017

Build is a massive conference for developers who build apps for Windows, Office 365, Edge/IE, SQL Server, Azure, Xbox, and HoloLens, using tools such as Visual Studio, Visual Studio Code, ASP.NET vNext, and product-specific SDKs and APIs. Build also features content for Android, iOS, and mobile web developers. Major updates to the Windows OS and .NET ecosystem are typically announced at this conference.

Videos of past years' sessions are available online.

Who should attend: Windows developers, .NET developers, SQL Server users, Azure PaaS users, and mobile developers (iOS, Windows Mobile, Android, and web)

Internet of Things World

Twitter: @IoTWorldSeries / #IoTWorld
Web: tmt.knect365.com/iot-world
Date: May 14-17
Location: Santa Clara, California, USA
Cost: Depending on when you purchase tickets, prices range from $795 to $3,195, Free tickets are available to attend the expo, which includes more than 300 exhibitors

Internet of Things World features content to meet the advanced technical needs, as well as tracks focused on utilities, energy, smart buildings, and construction. The event also includes a hackathon and Project Kairos Startup City, which will run over two days and feature over 100 cutting-edge startups.

Who should attend: Developers, IT executives, business executives, device manufacturers, transportation companies, supply chain and logistics professionals, sensor and embedded system experts, telecom professionals, and systems integrators

Google I/O

Twitter: @googledevs
Web: events.google.com/io
Date: May 16-18 (unconfirmed)
Location: Mountain View, California, USA
Cost: 2017 costs: $1,150 regular, $375 for full-time students (source)

First held in 2008, Google I/O, has become one of the most important developer conferences in the world. Like Apple’s WWDC, Google I/O isn’t strictly about mobile, but the event is heavily focused on Android and its ecosystem. The conference also covers developer tools and APIs for other Google products, services, and platforms, including its enterprise cloud platform, consumer online services such as Google Play, products for publishers and advertisers such as AdSense and Analytics, consumer devices such as the Cardboard virtual reality headset, and even some of the company’s “moonshot” projects.

Videos of past years' sessions are available online.

Who should attend: Developers working with Android or with the growing variety of Google web services, mobile apps, and hardware

June

Internet of Things Developers Conference

Twitter: @iotdevcon / #iotdevcon
Web: iot-devcon.com
Date: June 5-6
Location: Santa Clara, California, USA
Cost: N/A; $95-$495 in the past, depending on when purchased

Focused on solving the technical and business challenges of the IoT, the Internet of Things Developers Conference features an exhibit floor, in-depth technical sessions, tutorials, business strategy, and hands-on demos.

Who should attend: IoT developers

Fluent

Twitter: @fluentconf / @OReillyMedia / #FluentConf
Web: conferences.oreilly.com/fluent/fl-ca
Date: June 11-12, training; June 12-14, tutorials and conferences
Location: San Jose, California, USA
Cost: N/A

Fluent was held in 2012 as a new event for developers working with JavaScript, HTML5, and other web technologies. Since its launch, Fluent has expanded to include front end, back end, design, web performance, security, and more. Fluent offers a variety of forums, including immersive tutorials, expert sessions, and hallway tracks between sessions where attendees informally connect and share questions, knowledge, and perspectives with their peers.

Who should attend: Web designers and developers, including mobile and web infrastructure teams, JavaScript developers, architects, UI/UX designers, and system developers

Apple WWDC

Twitter: #WWDC18
Web: developer.apple.com/wwdc
Date: June (second or third week)
Location: San Jose, California, USA
Cost: $1,599

Final information on Apple's Worldwide Developers Conference usually isn't released until April, but before moving to San Jose in 2017, the event was held in San Francisco, and it could return there. The event is Apple’s biggest developer conference, so it attracts major attention from the press, industry analysts, Apple customers, and macOS/iOS developers. Although it’s not an exclusively mobile-focused conference, mobile dominates the proceedings. New versions of Xcode and App Store services are typically announced. 

If you’re unable to score a ticket, several independent events occur simultaneously in the area. One of the better-known examples is AltConf, which is free.

Who should attend: iOS and MacOS developers

AltConf

Twitter: @AltConference
Web: altconf.com
Date: June (second or third week)
Location: San Jose, California, USA (unconfirmed)
Cost: Free

Billed as the alternative Apple developer conference, AltConf is a community-driven event, assembled to serve developers and a product-driven community alongside Apple's WWDC. Although most of the presentations are focused on Apple’s ecosystem, other topics include design, business, marketing, and team management.

Who should attend: iOS and MacOS developers

Sensors Expo & Conference

Twitter: @SensorsExpo / #Sensors18
Web: sensorsexpo.com
Date: June 26-28
Location: San Jose, California, USA
Cost: Early-bird rates (through April 20) range from free expo hall passes to VIP conference passes for $1,199 ($1,399 April 21 through June 22, $1,599 on-site)

For more than 30 years, the Sensors Expo & Conference has focused exclusively on current and upcoming sensors and sensor-integrated systems. More recently, the conference added a new dimension with the emergence of the IoT, and it remains the top conference for learning about sensors from the industry's top experts.

Who should attend: Software engineers, scientists, researchers, academics, investors, and corporate buyers

July

AnDevCon

Twitter: @AnDevCon / #AnDevCon
Web: andevcon.com
Date: Mid-July (unconfirmed)
Location: Previous venues were in San Francisco, California, and Washington, DC.
Cost: Depending on when purchased, ticket prices typically range from free exhibit-only passes to all-access passes for $795

AnDevCon aims at developers building Android apps by providing training sessions, tutorials, and classes. All sessions deliver practical knowledge, with less high-level fluff and fewer inspiration-focused talks than other conferences.

Who should attend: Android developers

August

360 iDev

Twitter: @360iDev / #360iDev
Web: 360idev.com
Date: August 26-29
Location: Denver, Colorado, USA
Cost: Prices range from $799 to $1,049 (military and student discounts available)

360 iDev positions itself as the smaller, more intimate, independent alternative to Apple’s WWDC. Typically, around 400 people attend the event, which includes roughly 50 sessions and 40 speakers. It's the largest, longest-running independent iOS/Mac conference in the world, so it attracts some of the best and brightest in the industry as both speakers and attendees.

Who should attend: iOS and MacOS developers

September

DroidConNYC

Twitter: @droidconNYC / #DCNYC17
Web: droidcon.nyc
Date: September (unconfirmed).
Location: New York City, New York, USA
Cost: $195-$1,995 (unconfirmed)

The DroidCon global conference series serves the Android developer ecosystem. DroidCon NYC, a two-day conference, features three speaker tracks and code labs. There are also dozens of other DroidCon events around the world, so check for conferences nearest you.

Who should attend: Android developers and engineers

October

Release Notes 2018

Twitter: @release_notes / #releasenotes
Web: 2018.releasenotes.tv (link not yet active)
Date: October (unconfirmed)
Location: Chicago, Illinois, USA (unconfirmed)
Cost: In 2017, prices for the event were $799 and $999, depending on when they were purchased and a companion ticket was $199

Release Notes is focused on the Apple ecosystem, especially the business of making apps. The conference features practical talks on a range of business and technical topics. It's an opportunity to network with developers, designers, marketers, and salespeople from all over the world, as well as to attend social activities. Release Notes speakers are challenged to bring something new and unique to their presentations based on their talents and backgrounds.

Who should attend: iOS and MacOS developers, marketers, salespeople, designers, and business managers

November

DroidConSF

Twitter: @droidconsf / #droidconsf
Web: sf.droidcon.com
Date: November (unconfirmed)
Location: San Francisco, California, USA
Cost: Unconfirmed

DrodConSF is part of a global conference series focused the Android developer ecosystem. The San Francisco event is a two-day conference that includes five tracks and a hackathon. You can find dozens of other DroidCon events around the world, so be sure to check and see which conferences are nearest to you.

Who should attend: Android developers and engineers

There you have it: the best mobile and IoT conferences of 2018, month by month. Review the options and make your choices soon: Prices may vary based on how early you register. And remember that hotel and travel costs are generally separate from the conference pricing

Which are your favorite conferences and why? Post your comments below, and let us know if there are any other relevant events or conferences we missed.

Image: Courtesy of Google]]> Tue, 02 Jan 2018 08:00:00 EST Linda Rosencrance Linda Rosencrance node/2244

The best mobile and IoT conferences of 2018

Build the future

Jan 2, 2018

Linda Rosencrance

The most life-changing products to come out over the next few years will almost certainly center around mobile, the Internet of Things (IoT), and voice-control technology. Will your product be one of them? That depends on whether you receive that spark of inspiration first—and then execute. Attending conferences can help you find that inspiration, and the knowledge you'll need to build the future's most exciting and useful inventions.

Here is our shortlist of the most popular mobile and IoT conferences in 2018. We've listed them all, although not all dates, locations, and pricing were available at publication time—especially for those events taking place later in 2018. In those cases, we have provided historical information on the event to give you an idea of what to expect and what you'll get out of attending. Keep checking back: We'll update this guide as more information becomes available.

World Quality Report 2017-18: The state of QA and testing

GET REPORT

January

IoT Evolution Expo

Twitter: @IoTEvolution / #IoTEvolution
Web: iotevolutionexpo.com
Date: January 22-25
Location: Orlando, Florida, USA
Cost: Prices range from free for an Expo Plus Pass when purchased in advance ($100 at the expo) to $3,595 ($3,695 at the show) for the Diamond Group Plan, which grants admission to all proceedings for three of your company's employees

IoT Evolution Expo focuses on how IoT can drive business transformation in all industries through improved operational efficiencies, revenue opportunities, and problem solutions. It features track sessions, an exhibit floor, case studies, special events, networking opportunities, and other events.

Who should attend: Developers, IT executives, business executives, device manufacturers, transportation companies, supply chain and logistics professionals, sensor and embedded system experts, and systems integrators

CES

Twitter: @CES
Web: cesweb.org
Date: January 9-12
Location: Las Vegas, Nevada, USA
Cost: Prices range from $300 to $1,700. Early-bird discounts available

This legendary massive consumer electronics conference and expo covers a wide range of topics, including mobile- and IoT-related sessions on security, digital entertainment, e-commerce, gaming, robotics, storage, education technology, mobile apps, and networking professionals.

Who should attend: Anyone interested in the latest and greatest consumer electronics

February

DeveloperWeek

Twitter: @DeveloperWeek / #DevWeek18
Web: developerweek.com
Date: February 3-7
Location: Oakland, California, USA
Cost: Prices range from $35 for an Expo Pass to $1,999 for a DevExec World Pass

According to the conference organizers, DeveloperWeek 2018 is the world’s largest developer expo and conference series, with over 8,000 participants across the DeveloperWeek 2018 Conference & Expo, the DeveloperWeek Hackathon, the Official Hiring Mixer, and citywide partner events.

Who should attend: Software engineers, entrepreneurs, and venture capitalists

Mobile World Congress

Twitter: @GSMA / #MWC18
Web: mobileworldcongress.com
Date: February 26-March 1
Location: Barcelona, Spain
Cost: Prices range from an Exhibition Pass, at €799, to a Platinum Pass, at €4,999

Described as a gathering for the entire mobile industry, the MWC is owned by GSMA, an industry group made up of 800 mobile operators and 300 mobile ecosystem companies. It's a big conference: In 2017, the MWC in Barcelona drew more than 108,000 attendees from 208 countries, and more than 2,300 exhibitors. Many smartphone and mobile device launches happen here.

Who should attend: App developers, operators, equipment vendors, and financial, marketing, and entertainment industry professionals

Embedded World Exhibition and Conference

Twitter: @embedded_world / #ew18
Web: embedded-world.eu/home.html
Date: February 27-March 1
Location: Nuremberg, Germany
Cost: Conference blocks range from €345 to €850 (VAT included); classes are €410 for a half-day and €610 for a full day

The Embedded World Conference is designed for developers and designers of embedded systems. Organizers say it is Europe's biggest conference devoted to embedded systems development, and it addresses all major topics in this sector, featuring papers and classes, with a focus on concrete solutions.

Who should attend: Developers and designers of embedded systems

March

Industry of Things World USA 2018

Twitter: @IoTClan / #IoTClan
Web: industryofthingsworldusa.com
Date: March 7-9
Location: San Diego, California, USA
Cost: Pricing options are Industry Delegate Premium, $2,695; Industry Delegate, $2,295; and Solution Provider, $3,495. Group discounts are available

Organizers call this conference "the leading Industrial IoT event in the USA." This event focuses on the impact of the industrial IoT on current business models and production processes across all major industries. It includes topics such as automation, machine-to-machine communication, interoperability, analytics, and new business models. Part of the Industry of Things World global event series, this conference has become the meeting point for senior executives who want to learn more about the industrial Internet.

Who should attend: IoT specialists and strategists, IoT novices, cloud computing adopters, big data analytics experts, and anyone else involved in a business's digital transformation

SXSW (South by Southwest)

Twitter: @sxsw / #SXSW
Web: sxsw.com/schedule
Date: March 9-18
Location: Austin, Texas, USA
Cost: Prices range from $445 to $1,450

While music and film are key elements of SXSW, the event also has a strong technology component. Topics include startups, blockchain, health IT, artificial intelligence, IoT, virtual reality, augmented reality, smart cities, digital media, software design and development, open source, mobile design, and user experience.

Who should attend: Developers, founders, and embedded developers

OpenIoT Summit

Twitter: @EventsLF / #openiot
Web: events.linuxfoundation.org/events/openiot-summit
Date: March 12-14
Location: Portland, Oregon, USA
Cost: Prices range from $200 to $850, depending on when you register

Created by technologists for technologists, OpenIoT Summit delivers the technical knowledge attendees need to deliver smart, connected products that take advantage of rapidly evolving IoT technologies. It is the only IoT event focused on the development of open IoT products, according to organizers. Experts from the world’s leading companies and biggest open-source projects will present on what you need to know to bring smart, connected products and solutions to market.

Videos of past years' sessions are available online.

Who should attend: Firmware developers, software developers, application developers, and system architects

DroidCon Boston 2018

Twitter: @droidconbos / #DroidconBos
Web: droidcon-boston.com
Date: March 26-27
Location: Boston, Massachusetts, USA
Cost: Prices: $99-$249

DroidCon is a global conference series focused on the Android developer ecosystem. The Boston conference is a two-day, full-immersion tech event driven by local communities and influencers. You can find dozens of other DroidCon events around the world, so be sure to check and see if there's a conference coming up near to your home.

Who should attend: Android developers and engineers

IoT6 Exchange

Twitter: @nGage_IoT / #IoT6
Web: www.iot6exchange.com
Date: March 27-29
Location: Ponte Vedra Beach, Florida, USA
Cost: An invitation-only event, Contact organizers for information

IoT6 Exchange features an in-depth agenda with keynote sessions, end-user panels, breakout sessions, and case studies. Topics include the industrial IoT, the IoT, machine learning, artificial intelligence, robotics and smart engineering, and asset and fleet monitoring and tracking. Complimentary airfare, accommodations, meals, and registration costs are provided to qualified attendees in appreciation for time spent out of office. Attendees must inquire to be eligible to receive a hosted invitation.

Who should attend: Operations, logistics, and manufacturing executives and decision makers

April

RWDevCon 2018

Twitter: @RWDevCon / #rwdevcon
Web: rwdevcon.com
Date: April 5-7
Location: Alexandria, Virginia, USA
Cost: Prices range from $999 to $1,999

RWDevCon is an Apple iOS conference focused on high-quality programming tutorials. Located just outside Washington, DC, RWDevCon is put on by teams at raywenderlich.com and 360iDev. Content focuses on hands-on tutorials, where attendees can code along with the presenter. 

Videos of past years' tutorials can be purchased for $199.

Who should attend: iOS and MacOS developers

Internet of Things Applications Europe

Twitter: Not available
Web: idtechex.com/internet-of-things-europe/show/en/
Date: April 11-12
Location: Berlin, Germany
Cost: Ranges from €99 for an exhibition pass to packages as high as €2,995.

Internet of Things Europe is collocated with the Wearable Europe and Sensors Europe events, and a ticket to any conference grants attendees access to the others. This event addresses real business opportunities for the IoT, not hype. Business models, case studies, opportunities, and profitability are all covered. It features specific market verticals, in addition to emerging technologies.

Who should attend: IoT developers, IT pros, and CxOs

AppBuilders

Twitter: @appbuilders_ch / #appbuilders18
Web: appbuilders.ch
Date: April 16-17
Location: Lugano, Switzerland
Cost: Prices range from $299 to $399, depending on when purchased

AppBuilders is a mobile technology conference featuring sessions for both iOS and Android developers. Conference speakers have featured lead iOS and Android developers from high-profile companies, including Viacom and The New York Times.

Who should attend: iOS and Android developers

May

Microsoft Build

Twitter: @msdev / #msbuild
Web: build.microsoft.com
Date: May (unconfirmed)
Location: Seattle, Washington, USA (unconfirmed)
Cost: $2,195 in 2017

Build is a massive conference for developers who build apps for Windows, Office 365, Edge/IE, SQL Server, Azure, Xbox, and HoloLens, using tools such as Visual Studio, Visual Studio Code, ASP.NET vNext, and product-specific SDKs and APIs. Build also features content for Android, iOS, and mobile web developers. Major updates to the Windows OS and .NET ecosystem are typically announced at this conference.

Videos of past years' sessions are available online.

Who should attend: Windows developers, .NET developers, SQL Server users, Azure PaaS users, and mobile developers (iOS, Windows Mobile, Android, and web)

Internet of Things World

Twitter: @IoTWorldSeries / #IoTWorld
Web: tmt.knect365.com/iot-world
Date: May 14-17
Location: Santa Clara, California, USA
Cost: Depending on when you purchase tickets, prices range from $795 to $3,195, Free tickets are available to attend the expo, which includes more than 300 exhibitors

Internet of Things World features content to meet the advanced technical needs, as well as tracks focused on utilities, energy, smart buildings, and construction. The event also includes a hackathon and Project Kairos Startup City, which will run over two days and feature over 100 cutting-edge startups.

Who should attend: Developers, IT executives, business executives, device manufacturers, transportation companies, supply chain and logistics professionals, sensor and embedded system experts, telecom professionals, and systems integrators

Google I/O

Twitter: @googledevs
Web: events.google.com/io
Date: May 16-18 (unconfirmed)
Location: Mountain View, California, USA
Cost: 2017 costs: $1,150 regular, $375 for full-time students (source)

First held in 2008, Google I/O, has become one of the most important developer conferences in the world. Like Apple’s WWDC, Google I/O isn’t strictly about mobile, but the event is heavily focused on Android and its ecosystem. The conference also covers developer tools and APIs for other Google products, services, and platforms, including its enterprise cloud platform, consumer online services such as Google Play, products for publishers and advertisers such as AdSense and Analytics, consumer devices such as the Cardboard virtual reality headset, and even some of the company’s “moonshot” projects.

Videos of past years' sessions are available online.

Who should attend: Developers working with Android or with the growing variety of Google web services, mobile apps, and hardware

June

Internet of Things Developers Conference

Twitter: @iotdevcon / #iotdevcon
Web: iot-devcon.com
Date: June 5-6
Location: Santa Clara, California, USA
Cost: N/A; $95-$495 in the past, depending on when purchased

Focused on solving the technical and business challenges of the IoT, the Internet of Things Developers Conference features an exhibit floor, in-depth technical sessions, tutorials, business strategy, and hands-on demos.

Who should attend: IoT developers

Fluent

Twitter: @fluentconf / @OReillyMedia / #FluentConf
Web: conferences.oreilly.com/fluent/fl-ca
Date: June 11-12, training; June 12-14, tutorials and conferences
Location: San Jose, California, USA
Cost: N/A

Fluent was held in 2012 as a new event for developers working with JavaScript, HTML5, and other web technologies. Since its launch, Fluent has expanded to include front end, back end, design, web performance, security, and more. Fluent offers a variety of forums, including immersive tutorials, expert sessions, and hallway tracks between sessions where attendees informally connect and share questions, knowledge, and perspectives with their peers.

Who should attend: Web designers and developers, including mobile and web infrastructure teams, JavaScript developers, architects, UI/UX designers, and system developers

Apple WWDC

Twitter: #WWDC18
Web: developer.apple.com/wwdc
Date: June (second or third week)
Location: San Jose, California, USA
Cost: $1,599

Final information on Apple's Worldwide Developers Conference usually isn't released until April, but before moving to San Jose in 2017, the event was held in San Francisco, and it could return there. The event is Apple’s biggest developer conference, so it attracts major attention from the press, industry analysts, Apple customers, and macOS/iOS developers. Although it’s not an exclusively mobile-focused conference, mobile dominates the proceedings. New versions of Xcode and App Store services are typically announced. 

If you’re unable to score a ticket, several independent events occur simultaneously in the area. One of the better-known examples is AltConf, which is free.

Who should attend: iOS and MacOS developers

AltConf

Twitter: @AltConference
Web: altconf.com
Date: June (second or third week)
Location: San Jose, California, USA (unconfirmed)
Cost: Free

Billed as the alternative Apple developer conference, AltConf is a community-driven event, assembled to serve developers and a product-driven community alongside Apple's WWDC. Although most of the presentations are focused on Apple’s ecosystem, other topics include design, business, marketing, and team management.

Who should attend: iOS and MacOS developers

Sensors Expo & Conference

Twitter: @SensorsExpo / #Sensors18
Web: sensorsexpo.com
Date: June 26-28
Location: San Jose, California, USA
Cost: Early-bird rates (through April 20) range from free expo hall passes to VIP conference passes for $1,199 ($1,399 April 21 through June 22, $1,599 on-site)

For more than 30 years, the Sensors Expo & Conference has focused exclusively on current and upcoming sensors and sensor-integrated systems. More recently, the conference added a new dimension with the emergence of the IoT, and it remains the top conference for learning about sensors from the industry's top experts.

Who should attend: Software engineers, scientists, researchers, academics, investors, and corporate buyers

July

AnDevCon

Twitter: @AnDevCon / #AnDevCon
Web: andevcon.com
Date: Mid-July (unconfirmed)
Location: Previous venues were in San Francisco, California, and Washington, DC.
Cost: Depending on when purchased, ticket prices typically range from free exhibit-only passes to all-access passes for $795

AnDevCon aims at developers building Android apps by providing training sessions, tutorials, and classes. All sessions deliver practical knowledge, with less high-level fluff and fewer inspiration-focused talks than other conferences.

Who should attend: Android developers

August

360 iDev

Twitter: @360iDev / #360iDev
Web: 360idev.com
Date: August 26-29
Location: Denver, Colorado, USA
Cost: Prices range from $799 to $1,049 (military and student discounts available)

360 iDev positions itself as the smaller, more intimate, independent alternative to Apple’s WWDC. Typically, around 400 people attend the event, which includes roughly 50 sessions and 40 speakers. It's the largest, longest-running independent iOS/Mac conference in the world, so it attracts some of the best and brightest in the industry as both speakers and attendees.

Who should attend: iOS and MacOS developers

September

DroidConNYC

Twitter: @droidconNYC / #DCNYC17
Web: droidcon.nyc
Date: September (unconfirmed).
Location: New York City, New York, USA
Cost: $195-$1,995 (unconfirmed)

The DroidCon global conference series serves the Android developer ecosystem. DroidCon NYC, a two-day conference, features three speaker tracks and code labs. There are also dozens of other DroidCon events around the world, so check for conferences nearest you.

Who should attend: Android developers and engineers

October

Release Notes 2018

Twitter: @release_notes / #releasenotes
Web: 2018.releasenotes.tv (link not yet active)
Date: October (unconfirmed)
Location: Chicago, Illinois, USA (unconfirmed)
Cost: In 2017, prices for the event were $799 and $999, depending on when they were purchased and a companion ticket was $199

Release Notes is focused on the Apple ecosystem, especially the business of making apps. The conference features practical talks on a range of business and technical topics. It's an opportunity to network with developers, designers, marketers, and salespeople from all over the world, as well as to attend social activities. Release Notes speakers are challenged to bring something new and unique to their presentations based on their talents and backgrounds.

Who should attend: iOS and MacOS developers, marketers, salespeople, designers, and business managers

November

DroidConSF

Twitter: @droidconsf / #droidconsf
Web: sf.droidcon.com
Date: November (unconfirmed)
Location: San Francisco, California, USA
Cost: Unconfirmed

DrodConSF is part of a global conference series focused the Android developer ecosystem. The San Francisco event is a two-day conference that includes five tracks and a hackathon. You can find dozens of other DroidCon events around the world, so be sure to check and see which conferences are nearest to you.

Who should attend: Android developers and engineers

There you have it: the best mobile and IoT conferences of 2018, month by month. Review the options and make your choices soon: Prices may vary based on how early you register. And remember that hotel and travel costs are generally separate from the conference pricing

Which are your favorite conferences and why? Post your comments below, and let us know if there are any other relevant events or conferences we missed.

World Quality Report 2017-18: The state of QA and testing

GET REPORT

Image: Courtesy of Google

TechBeacon's top 10 DevOps stories of 2017 highlight the evolving conversation around DevOps, and identify the topics that mattered most both to practitioners and to managers. The list includes stories on the challenges of DevOps adoption, lessons from the front lines, tips for finding the best skills for the job, advice on how to tell if your DevOps and continuous integration (CI) processes are really working, and guidance on how to secure your DevOps environment.

DevOps Enterprise Summit 2017 top takeaway: Adoption rapidly moving beyond IT

One of the key takeaways from the DevOps Enterprise Summit 2017 conference in San Francisco was DevOps' evolution from something of a curiosity to a mainstream process in most organizations. Adoption of DevOps has moved beyond IT, with people joining the conversation from across the organization, including in finance, HR, and information security functions, says Electric Cloud's vice president of marketing, Sam Fell. Here he lists the key takeaways from the event.

DevOps study finds informal teams perform better

There's clearly something to be said for informality when it comes to DevOps processes. An HPE survey on process maturity among enterprises engaged with DevOps methodologies showed that organizations that enjoy the most success with DevOps have the least formal approaches. Organizations at the lowest levels of process maturity are releasing code the fastest, and with the highest quality. Micro Focus technology evangelist John Jeremiah takes a closer look at research findings, and explains why the results may not be as paradoxical as they appear.

Lessons learned from 6 high-profile DevOps journeys

More companies are turning to DevOps to develop their applications, but not everyone is seeing increased productivity and profitability. So what are the ingredients of truly successful DevOps transformations? Chris Null looks at six high-profile initiatives that succeeded at Target, New Relic, Microsoft’s Visual Studio, Walmart, Netflix, Walt Disney, and Capital One—and what they all have in common.

10 questions to ask every DevOps team candidate

Having a good DevOps team is fundamental to your ability to derive gains such as greater application stability and faster code delivery from your development environment. To succeed at DevOps, you need professionals who can balance coding skills with CI/CD competencies. But do you know how to find them, or what questions to ask to separate the best from the rest? Jennifer Zaino discovers the 10 questions every manager should ask before hiring a DevOps professional.

How value-stream mapping delivers a better DevOps toolchain

Enterprises that use DevOps tools to enable continuous build, test, and delivery environments need a way to measure the effectiveness of those tools in uniting segmented workgroups and disciplines within the organization. For that to happen, you need to integrate DevOps toolchains, gain visibility and traceability across the development lifecycle, and correlate data between existing tools. CollabNet's general manager of DevOps, Eric Robertson, explains the benefits of value-chain mapping.

Are you really doing continuous integration? Here's how to tell

Many organizations think they are practicing CI when actually they're are not. Just because you have a CI server that your developers regularly use to check if their code has passed, or you deploy more than once a quarter, does not mean you are doing CI or CD. Suzie Prince, head of product at ThoughtWorks, explains what good CI really means, and what it looks like.

30 common challenges to DevOps and how to resolve them

DevOps may have become mainstream, but many teams continue to struggle with pretty fundamental questions about where to start, what challenges they are likely to face, and how to address those challenges. A lot of the uncertainty stems from the shift in focus from development to delivery that DevOps entails. IBM's worldwide lead for DevOps adoption, Peter Eeles, walks you through 30 of the most common DevOps challenges teams face, and offers tips on how to address each.

5 soft skills of successful DevOps software engineers

Software engineers need more than just an understanding of programming languages and automation tools to succeed at DevOps. Soft skills such as empathy, integrity, resiliency, and the ability to communicate with others are becoming critical job requirements as well in many organizations. Mike Kail, Cybric's chief technology officer, describes the five soft skills that are critical to success as a DevOps engineer.

30 essential container technology tools and resources

Software containers are a great tool for deploying complete application environments quickly—sometimes in a matter of seconds, depending on the completeness and size of container packages. But do you know what standards to use and how to deploy the images, store old versions, and manage them? Do you have a handle on the combination of products and services you need to build and manage containers optimally in your environment? Excelon Development's managing consultant, Matthew Heusser, reviews the 30 essential container technologies you should consider.

Is your application architecture prepared for microservices?

One goal of DevOps is to enable small, incremental improvements to applications quickly, frequently, and without disruption. A microservices architecture is ideal for supporting this goal, but most organizations are saddled with tightly coupled, monolithic applications that aren't easy to decompose. Technology journalist Ericka Chickowski reports on why that shouldn't stop your organization from starting out on its DevOps journey anyway.

TechBeacon's top 10 application development stories for 2017 highlight where the jobs are, the trends and best practices to follow, the habits to avoid, and the things you need to read up on to stay on top of the game. Collectively, these stories illuminate the opportunities and challenges facing app developers, and best practices for remaining successful.

The best-paying programming languages

Programming jobs command relatively high salaries in the current job market, but the language you choose can make a difference in how much you make. Not all programming languages command the same salaries. Programmers with experience in languages such as Scala and Clojure often command higher salaries than those with PHP, Go or Visual Basic skills. Dave Fecak, principal of Fecak Inc., lists the hot and the not-so-hot programming languages.

6 code and framework trends you should follow in 2017

Keeping on top of the latest and greatest in the programming ecosystem can be challenging. The sheer number of languages and frameworks out there makes it hard to keep track of the trends you should follow, the ones you should keep an eye on, and the ones you can safely ignore. TechBeacon managing editor Mitch Pronschinske has done the legwork for you in this rundown.

How to master Android: What developers can learn from 21 apps

Open-source apps can be great resources for developers who want to learn and improve their skills. The apps provide an opportunity for every programme, from  beginner to expert, a way to get hands-on experience with code before diving into it more fully. Treebo Android developer Aritra Roy offers his list of 21 open-source Android apps you can use to hone your skills. The apps run the gamut from those that beginners can benefit from to those designed for experienced Android developers.

35 programming habits that make your code smell

There are things that all developers do and practices software engineers use that undermine the quality of their work. Sometimes the habits are inadvertent, sometimes not. Most mistakes programmers make can be categorized under code organization, teamwork, code writing, and testing, says senior web developer Christian Maioli. Here's his list of the 35 most egregious programmer fails across the categories.

Lessons from 7 highly successful software engineering cultures

Some of the most successful technology companies allow a great deal of autonomy and freedom to innovate in the workplace. Giants such as Facebook, Netflix, and scores of other technology firms didn't get where they are by following others; they set out to be different. These seven technology firms have established cultures that inspire and motivate people to explore new possibilities, says Mitch Pronschinske in this look at their innovative cultures.

4 forgotten code constructs: Time to revisit the past?

A programming language is only as good as the programmers who code in it. The tendency of many programmers to ignore the right way in favor of the popular way can reduce code clarity and readability and lengthen development time, says senior web developer Christian Maioli. He argues for using four old—and mostly forgotten— code constructs.

The top 12 international cities for software engineers

Silicon Valley is the heart of the US technology industry, but that doesn't mean it's the only place to be if you are a software engineer. There are plenty of other cities around the world to practice software development if you are looking for a change of scene or planning a move to another country. TechBeacon's Mitch Pronschinske lists 12 cities you'd definitely want on your shortlist, along with a cost-of-living breakdown and data on crime, safety, and pollution.

Bootcamps won't make you a coder. Here's what will

Don't get fooled by the job placement numbers that those three- and six-month coding bootcamps throw at you. Close examination of bootcamp numbers show they can be misleading, they're not audited, and often they're just plain inflated. Bootcamp attendees complain about poor quality of training,  improperly trained staff, and that the trainers don't spend nearly as much time as needed on core coding practices. TechBeacon managing editor Mitch Pronschinske takes a skeptical look at some of the facts behind the bootcamps' claims and offers some DIY alternatives that let you teach yourself to code.

8 best practices for microservices app sec

Implementing a microservices architecture can free you from some of the limitations of monolithic approaches at a time when speed and flexibility have become critical considerations for app development, but microservices architectures can introduce a whole new set of security issues. Senior software engineer Marco Troisi examines some of the keys ones, and recommends nine best practices for addressing them.

A guide to RESTful API design: 35+ must-reads

Representational state transfer (REST), has become the de facto standard in recent years for designing APIs that run over HTTP. But do you understand all of the nuances of RESTful API design? How well do you really comprehend concepts such as resource naming, HTTP method usage, hypermedia, and versioning? To get you up to speed, consultant Bill Doerrfeld compiled this useful list of more than 35 must-read resources on API design.

Our collection of top stories for 2017 combines a mix of guidance, advice, and caveats on some of the most important topics around agile development practices today. Among them are stories on project management practices that work, what you can do to get executives on board with agile, hacks for getting distributed agile right, and the importance of "Modern Agile" and "Heart of Agile" practices.

Why hybrid agile-waterfall projects fail

Software projects fail for all sorts of reasons. But those that combine agile and waterfall methodologies tend to have higher failure rates than pure agile projects. Agile consultant Yvette Francino reviews the conflicting findings of two separate surveys to explain why so many organizations still take the hybrid approach anyway, and what they can do to improve their chances of success.

Why your execs don't get agile and what you can do about it

Senior management executives often have a tendency to undermine the very agile projects they champion because of an incomplete understanding of the process and what it entails. Excelon Development managing consultant Matthew Heusser offers the lowdown on the five most common mistakes executives make when it comes to agile development, including a tendency to view agile as a team activity and a penchant for over-focusing on the DevOps aspect.

Modern Agile and Heart of Agile: A new focus for agile development

"Modern Agile" and "Heart of Agile" are two movements that emerged recently in response to the general disillusionment around agile and Scrum. Both intend to change agile practices by making them simpler and focused on four key principles. Agile consultant Yvette Francino explains why Modern Agile and Heart of Agile are more than just marketing hype.

Project management: A surefire way to kill your software product

Software development is not really a predictive or repetitive activity. Imposing a project management structure on the process only complicates matters and detracts from the overall quality of the final product. The constant innovation and learning required for software development is totally incompatible with project management's focus on predictability and efficiency, explains ThoughtWorks' Steven Lowe.

A practical guide to user story splitting for agile teams

Splitting project features into smaller stories can help maintain the cadence of an agile release. But doing it right requires a keen understanding of story-splitting techniques and best practices for story sizes. In this practical guide, Mark Balbes, vice president of architecture at WWT Asynchrony Labs, offers tips on how to get story splitting for agile teams right every time.

Being agile and working smart are not the same thing

Development focus can change throughout a project, so adaptive thinking is essential to the effectiveness of your process. Rather than getting hung up on the best methodology for a particular project, know your team's strengths and weaknesses, when it makes sense to introduce structure into the process, and when it does not. Senior web developer Christian Maioli makes a case for why being agile is not the same as being smart.

Distributed agile teams: 8 hacks that make them work

Making agile work in a distributed environment can be tough, but it can happen. The realities of geographic expansion, mergers, offshoring, and the current job market often require organizations to have a distributed team for agile. Not sure how to do it? Kurt Bittner, vice president of enterprise solutions at Scrum.org, lists eight hacks for making distributed agile work.

The seduction of the two-week sprint

Just because everybody is doing two-week sprints doesn't mean you should also. Two-week iterations might work well for some teams, but they're not for everyone. Some of the common arguments in favor of short iterations, in fact, ignore challenges that teams are likely to face in practice. Blue Agility's agile transformation specialist Anthony Crain explains why mandating two-week sprints for all teams in your organization may not be a great idea.

Why agile teams need to share the product owner role

Making the product owner (PO) solely responsible for defining, interpreting, and prioritizing requirements is at odds with other agile practices, such as collaborative swarming and cross-functional teams. Instead of building a box around the PO function, consider ways to share elements of the PO role with the rest of the team. Comcast senior engineering director Stephen Frein explains how.

Tailoring SAFe: It's not a one-size-fits-all framework

The Scaled Agile Framework (SAFe) is designed to provide better governance, coordination, and consistency for agile teams that have hundreds, or thousands, of members. But thinking of it as a one-size-fits-all approach for large teams goes against the very fundamentals of the Agile Manifesto, says Excelon managing consultant Matthew Heusser. Here, he offers advice and perspectives from four veteran practitioners in the field.

What are the top issues in agile your team is facing? We'd love to get experts sharing their experiences in 2018, so let us know in the comments below.

Given the current state of cyber threats, you might think that hiring managers are doing all they can to secure top cyber talent when recruiting for these positions. After all, the demand for cybersecurity pros is at an all-time high, supply is failing to keep up: According to projections, according to our research study “The Cyber Security Hiring Crisis." The supply of cybersecurity pros will outpace the number of open positions by 1.8 million.

Nonetheless, many organizations today treat cyber jobs just as they do IT jobs in terms of compensation and benefits, even though the roles are completely different.

That's wrong. Here's why.

Inconsistency in job titles and functions, and no common language

Currently, 35 cybersecurity job categories fall under the cyber umbrella, but actual roles don't necessarily carry the same job titles or have the same responsibilities from one organization to another. Roles within cybersecurity vary widely, and each has a unique set of skills.

Hiring managers need to understand the core responsibilities for each position prior to defining the job title, and each job title must be consistent with industry-recognized titles if you expect to attract the attention of job seekers—and optimize for search engines.  

Reliance on unreliable data

Reports from national institutions designed to standardize salaries are often old, and fail to recognize the full range of cyber jobs. For example, the US Bureau of Labor Statistics provides salary data for cyber professionals, but only for the role of “Information Security Analyst.”

In this fast-paced world, the data changes all the time So if a report is even a month old, it will inaccurately reflect industry salaries. This can create misunderstanding about the role and the appropriate compensation.

Cybersecurity is a diverse field with many roles, and the use of outdated data is far from adequate to empower hiring managers and internal recruiting teams to generate competitive offers. Services such as Glassdoor might have more up-to-date information, but their reliance on base salaries leaves a lot of information out, such as bonuses, benefits, stock options, and other creative compensation methods that make many of these jobs attractive to job seekers. Also, niche roles are underrepresented, with very little data available to assist hiring managers. 

Pushback from HR

As the middleman between hiring managers and candidates, HR often possesses a lot of power. The problem is that many HR professionals are uninformed about the special skills cybersecurity practitioners must have. Compounding the difficulty is the fact that most organizations are reluctant to give cyber professionals higher salaries than IT professionals, despite the dearth of talent and the highly specialized skills the role demands.

In our research study, mentioned above, we found that hiring managers are more likely to turn to their own internal networks, social media, and outside staffing firms to source candidates, rather than leave it in the hands of internal recruiters. Cybersecurity practitioners are highly skilled and specialized, and general recruiters often aren't equipped with the networks or the industry specialization needed to attract and retain this unique talent.

Change the mindset, change the game

Many organizations fail to realize that by not offering market rates for cybersecurity positions, they're sending a message that cybersecurity is not a priority in their organizations. In fact,  cybersecurity professionals say that the No. 1 reason they change jobs is that they feel that their profession is not a priority for the organization. Even is you can hire cybersecurity professionals under these circumstances, it creates a retention and attrition problem that nearly eclipses the recruiting challenges. Clearly, this outdated mindset must change.

Checking a compliance box is not enough to prepare for risk. Organizations must be willing to invest in the critical roles that will keep the business up and running as cyber threats continue to evolve. The best way to do this in today's highly competitive market is to offer top compensation and benefits to recruit—and retain—the talent you need to protect the business.

These tools are still unfamiliar to most IT leaders. But not only are such tools cost-effective, but low-code platforms offer the benefits of speed, the flexibility to address customer needs, and the ability to streamline business processes.

Speed that's cost-effective

Low-code development tools use a visual design format, rather than coding syntax, so designers not trained in computer science can use them to contribute to the development process. A user-friendly interface also lets designers get changes completed faster and more efficiently, and that saves time and money on labor resources.

But development professionals can also use low-code tools to deliver applications faster, and with fewer bugs. These tools use prebuilt libraries and functionality to accelerate development and maximize component reuse. About 70% of IT leaders found that low-code platforms are more affordable compared to traditional development platforms, according to a Forrester Research survey, and 80% cited the ability to meet requirements within budget.

The University of South Florida used a low-code development tool to create a communication platform for its faculty, staff, and students to communicate following Hurricane Irma. "We needed to quickly create a solution that helped us gather data from people calling our campus information line and also direct them toward the proper resources," said Sidney Fernandes, Vice President IT and CIO at USF. 

The USF team deployed a platform to its community within hours by using existing applications and data points as building blocks to get the new app up and running in a fast and cost-effective way. "We already had building blocks in place, from authentication, to student and faculty information. Leveraging the existing infrastructure, we were able to develop a solution that worked seamlessly,” said Fernandes.

Flexibility to meet customer needs

In today's digital age, customers expect faster and more personalized experiences when dealing with any company. About 90% of IT leaders find that the flexible design of low-code platforms helps to significantly improve customer experience compared to traditional development platforms, Forrester found.

In a retail environment, low-code provides capabilities that allow users to rapidly implement new cost structures or operation system models. The ability to implement change is particularly important in retail during seasonal changes and holiday shopping peaks. 

On the consumer side, low-code tools also help retailers provide better customer service through omni-channel customer experiences and order fulfillment applications. Low-code platforms are helping companies to slash delivery times from months to weeks (or even hours), implement real-time updates, and deliver stronger applications driven by user feedback.

Taken together, this allows low-code users the ability to quickly scale and change applications to meet consumer demands.

Streamlining business processes

Beyond customer experience, companies are also recognizing the need to transform their business and operating models to increase revenues in the digital era.

Low-code can help automate the most critical and complex processes with drag-and-drop process modeling that allows users to build from existing applications and data. The built-in security features make low-code a reliable and secure solution for internal processes in industries such as financial services and healthcare that support critical infrastructures and maintain sensitive data assets.

A tool for digital transformation

Undergoing digital transformation is a multi-faceted journey, and there is no single tool or approach that will automatically make transformation a reality. A company's digital transformation strategy requires a wide range of new approaches and technologies. Low-code platforms can play a central role in this mix by reducing the time and money spent on traditional development processes and empowering enterprise employees to delve into development, regardless of their technical background.

Experian’s enormous ConsumerView database was publicly available in an Amazon S3 bucket owned by Alteryx, Inc. All 123 million records of it.

How can you avoid being next? In this week’s Security Blogwatch, we learn the lessons that others ignored.

Your humble blogwatcher curated these bloggy bits for your entertainment. Not to mention: EDM2017 …

What’s the craic? Thomas Fox-Brewster cunningly calls it extraordinary:

Information on more than 120 million American households was sitting … exposed.
…
It included an extraordinary range of personal details … 248 different data fields for each household. … The data was sitting in an Amazon Web Services storage "bucket," left open to anyone.
…
The data … was left online by marketing analytics company Alteryx. … The firm had purchased the information from Experian.
…
After being informed … the company took action and secured the database from public view last week. … Alteryx played down the severity of the leak: … "The file contained no names of any individuals or any other personal identifying information. … The information in the file does not pose a risk of identity theft to any consumers."
…
Experian [said] "This is an Alteryx issue, and does not involve any Experian systems."

Oh, no PII. So that’s okay then, right? Tara Seals disagrees:

As more and more collation takes place, across many different sources, these databases become fingerprints—troves of information that provide a startlingly complete picture of each individual. … It remains to be seen what information has fallen into bad actors’ hands.
…
The scale of the issue puts it in the running with the infamous Equifax incident, as it touches virtually every American household. … Exposed within the repository are massive data sets belonging to Alteryx partners Experian … and the US Census Bureau.
…
At issue is once again an Amazon Web Services S3 cloud storage bucket that was misconfigured and inadvertently left open to the public internet. [Which] could mean … organized fraud techniques like phantom debt collection, identity theft and security verification.

Who discovered it? UpGuard’s Chris Vickery, here channeled by Dan O'Sullivan:

The exposed data reveals billions of personally identifying details … about virtually every American household. From home addresses and contact information, to mortgage ownership and financial histories, to very specific analysis of purchasing behavior, [it’s] a remarkably invasive glimpse into the lives of American consumers.
…
The continuing concentration of data by a number of large enterprises … has not been accompanied by greater prudence and process. … Data exposures such as this are capable of exposing the vast majority of American households to compromise with one error.
…
While the spreadsheet uses anonymized record IDs to identify households, the other information [is] sufficiently detailed as to be not merely often identifying, but with a high degree of specificity. … This exposed data provides a highly detailed database of tens of millions of Americans’ personal, financial, and private lives.
…
Experian's ConsumerView information is proprietary, sold only to other enterprises; how do you ensure an external partner or vendor to whom you are entrusting your data in this way ensures it remains secure? … This is an enormous problem facing the IT landscape today. … Most enterprises lack the ability to even assess the security postures of external vendors.

Who’s first to comment? How about mikele11111?

Addresses don't constitute personally identifying information?
…
An address is almost as good as it gets next to a SSN when it comes to getting identifying details on someone.

Troy Hunt reckons education is a fundamentally sensible idea:

We have a data breach problem. [And] things are going downhill in a hurry.
…
You know the old "prevention is better than cure" idiom? Nowhere is it truer than with data breaches. … Every single one of them can be traced back to a mistake made by humans.
…
Part of the challenge here is that people simply don't know that there's a big part of their knowledge missing. … They have no idea that they have a massive deficiency in their competency.
…
Education is the best ROI on security spend. … It's cheap … has enormous upside [and] you leverage it over and over again.

Before coming up with his solution, Ray Knapp sleeps on it: [You’re fired—Ed.]

Send the entire board of these companies to jail, for a year.
 
This will immediately stop these breaches, every time there is a breach of personal data it should be automatic jail time no if ands or buts.

And RcouF1uZ4gsC is easy for you to say:

Personal data right now is considered an asset. It needs to be seen as a liability.
…
There must be a way to track from when a consumer input the data all the way through. Fraud in regards to this provenance is punishable by jail time.
 
If you have data that does not have provenance, the company will be severely fined and people will go to jail. In the event of any data breach, not only will the company that had the breach be [punished], all companies that provided the data to the company … that had the breach will also be punished.

But lima looks abroad:

In Europe, especially with the upcoming GDPR laws, data is seen as a liability by many companies.
…
[This] means that companies need to think twice before storing it — if the data is sufficiently valuable, companies will still store it, but they have to manage it properly and mitigate risks since penalties are harsh.

But what of Amazon Web Services? mcheshier suggestifies:

My biggest problem with S3 is the old multi-layered security model with bucket policies and ACLs.
 
They need to update it to just use IAM like everything else.

And here’s leonbev:

Amazon has been sending their customers warnings about misconfigured S3 buckets for awhile now. In order for something like this to happen, a customer would have ignored these warnings for the past 9 months.

So, yeah, someone probably deserves to be fired.

If true, gfody’s story is an astounding indictment: 

I worked at Experian and had access to ConsumerView by way of any number of crazy integration schemes … perpetually making a mockery of security protocols in order to meet deadlines and please clients.

Pretty much anyone with network access could download a copy and walk out with it.

Meanwhile, houghi looks on the bright side:

But it is good to know these companies will pay a hefty fine, right?
 
Right? Guys?

The moral of the story? Urgently review your cloud-storage policies, IAM, ACLs, etc. And if you share sensitive data with partners, how are you auditing that?

And finally …

Best EDM of 2017, according to DJs from Mars 

Bonus linkage: The History of EDM

You have been reading Security Blogwatch by Richi Jennings. Richi curates the best bloggy bits, finest forums, and weirdest websites … so you don’t have to. Hate mail may be directed to @RiCHi or sbw@richi.uk. Ask your doctor before reading. Your mileage may vary. E&OE.