This Is Why Bugs in Windows Updates Increased

In a recent video, Jerry Berg, a former Microsoft Senior SDET who worked at Microsoft for 15 years, explained why bugs in Windows updates increased in the past couple of years.

When he worked in Microsoft, Berg’s work was to design and develop tools and processes to automate testing for the Microsoft Windows operating system. But he left the company after Windows 8.1 shipped to the public. Now, he is better known as Barnacules.

Windows 10 V1903 KB4512941 Update Causes 100% CPU Core Usage

High CPU usage is a common problem that happens to PCs. The latest Windows 10 KB4512941 update also leads to this issue, but you can find solutions to fix it.

Read More

Testing Processes Changed in the Past Couple of Years

The reasons why bugs in Windows updates increased lie in the changed testing processes. In his video, Berg describes how testing was done in the late 2014 early 2015 period and how Microsoft’s testing processes changed since then.

Testing Processes in 2014/2015

In 2014/2015, Microsoft employed an entire team to test the operating system, builds, updates, drivers, and other codes. The team consisted of multiple groups that would run tests and discuss bugs and issues in daily meetings.

This team would run tests manually and through automated testing. Only if tests were passed, the code would be integrated into Windows. The team can get testing data from the following three sources:

1. Automated testing system. Through automated testing, the team ran the tests on “real” hardware in a lab. This machine had different types of hardware components so that it could cover a wide range of system configurations. But there was one defect in this process: only bugs affecting certain hardware components or configurations could be detected.
2. Microsoft employees. Microsoft employees could self-host Windows so that their machines would also be used for testing purposes. Through this testing mode, the testing team can get feedback from Microsoft employees when they encountered issues during work days.
3. Telemetry and Windows insiders. Windows Insider builds are installed on millions of devices and Microsoft collects Telemetry from all of these devices. If something crashes, Microsoft gets information about it.

How to Fix Microsoft Compatibility Telemetry High Disk Easily

This article introduces Microsoft Compatibility Telemetry. This post also teaches you how to disable it and how to solve high disk problem caused by it.

Read More

After getting testing data, Microsoft’s testing team would analyze bugs and issues, and then supply engineers with the data they required to resolve these.

Current Testing Processes

Nowadays, the testing processes have changed in the following aspects:

1. The Windows test team was laid off. Since Microsoft moved the focus from three different systems (Windows, Windows Mobile, and Xbox) to a single system, it laid off almost the entire Windows test team and moved most of the testing to virtual machines, which meant that tests were no longer conducted on real and diverse hardware configurations for the most part.
2. Self-hosting is not as widely used anymore as it was before.
3. Engineers rely more on Telemetry and Windows insiders. Nowadays, engineers collect bugs and issues through Telemetry. However, there is one problem associated with the collecting of Telemetry. That is, most bugs are not caught by it. If something does not work right, Microsoft may not be able to discern the relevant bits from Telemetry data.

There also exists a problem in Windows insider. In theory, users can report issues, but many of them may not report. Even if they report issues, these issues may go under because of other feedback that Microsoft gets from insiders.

Additionally, it is often the case that insiders don’t supply necessary information to Microsoft, which causes huge issues for the engineers tasked with resolving these issues.

After collecting issues, engineers look at Telemetry to figure out how to fix these issues and then push fixes to customer devices running Insider Builds again to see if the issue gets fixed or if it creates new bugs.

Result

Microsoft replaced the in-house testing team with Telemetry, and replaced many testing PCs with virtual environments. It only gathers testing data from Insider Builds pushed to consumer and business devices.

All of that led to an increased number of issues and bugs that customers face on production machines when installing Windows updates or feature updates.

Finally, to avoid undetected issues affecting a large number of customers, Microsoft stopped pushing out new feature updates to everyone at once and introduced gradual rollouts, which would prevent feature updates from being delivered via Windows Update to the majority of machines in the early days of the release.

Why Windows 10 Users Started Switching to Other Platforms

As time wears on, Windows 10 exposed to some obvious issues. So, you may wonder should I swift to other platforms? Here are the answers you want to get.

Read More