Category Archives: IT

Challenges of testing when using Scrum

Recently I dived into the merits and challenges of software development (or more generic IT products for that matter) in Agile based settings, in particular using Scrum.  Scrum is one of those methodologies that are part of a wave of Agile based methodologies (LEAN, extreme programming among others) that emerged in the 90’s and in 2001 united for the so called Agile Manifesto. Although Scrum is around for some time, it seems that in recent years Scrum has taken its place in countless organisations as the preferred method of developing software. While studying the Scrum approach I kept a focus on the inclinations for the testing process. I will not go into lengths to give a full description of the Scrum framework in this article but instead focus on possible weaknesses and strengths in relation to the testing process.

A clearly positive aspect of Scrum is that it is not hard to understand. Scrum is defined as a process framework, it does not prescribe an in depth approach on every aspect of the software development. Scrum contains a comprehensive workflow and its own taxonomy (with concepts like SprintIncrementDefinition of Done) is limited. The essential of the Scrum approach could be defined as: transparant software development based on a just-in-time workload, the effort to ensure all involved have a clear picture of the products that will be delivered and the benefit of a team based operation where the group effort is more important than individual success of the team member.

Because the scope of a development iteration (the Sprint) is limited (usually 2 weeks and maximum 4 weeks) the chances of deviation between the mutual understanding of the work to be done and the actual performed work could theoretically not be a too big surprise. Along the path of the Sprint there are mechanisms in place (Daily ScrumSprint Review and Introspective Review) to eliminate risks even further. As -in general- testing activities are based around inventorying and addressing the potential risks, the Scrum approach from a testing perspective is a step in the right direction to deliver quality products.

Yet besides these positive aspects, Scrum also will introduce some trade offs. Before diving into these trade offs, some perspective is needed on how to judge whether something might be considered a weakness or strength when we look at the Scrum process and its intertwining with testing. Over the past decades the most clear contraposition to the Scrum approach of software development has been the waterfall method. The waterfall method implies that there is strict sequential order of activities in the process: it is -for example- uncommon to start coding before the functional and technical requirements are completely set and approved by the stakeholders. An implication of this method is that the time to bring a product to the market might be taking a (too) long time. When the analysis phase within a waterfall project is carried out, internal or external (market) situations might have changed and/or the set requirements have changed. New requirements could be added, old ones could be not longer applicable. In an ever more competitive business world the rise of Agile based methods became new, interesting ways to overcome the waterfall method’s shortcomings. A less positive track record for timely delivered waterfall based projects also contributed to a rise in interest for agile methodologies and Scrum in particular.

Yet for testing purposes the waterfall method unmistakable has its advantages. The well known V-model for testing represents this.

A test strategy could be plotted against the deliverables (or project phases) in the waterfall based project. It gives the test manager time and opportunity to carefully plan and carry out the test strategy, changes never come too abrupt. And if changes come they could be easily adapted into a refined or updated test strategy.

In the Scrum world things work different. The Sprint is short cyclic iteration with a predefined outcome of the Sprint, usually defined as ‘working software’. This approach offers challenges in the testing process. First of all the testing effort has a tendency to become inefficient compared to a waterfall based approach. A test process needs to be in place within each Sprint leading to some inherent overhead (like writing test scripts, setting up test environment, inviting testers for the User Acceptance Test). It is still easier and less time consuming to perform a test process once for a larger number of tasks than numeral times with each Sprint for a specific or small number of tasks. The members of the Development Team (usually between 3 to 9 members) have no specific role within the team and should be able to take on the tasks of another team member. So a team member who develops software should also be able to test the software. This might lead to effects of tunnel vision (approving one’s own work) and a lack of segregation of duties.

The outcome of a Sprint does not necessarily has to be a piece of software that is implemented in the live environment. Preferably it is, but in the Definition of Done it also can be determined that the Scrum Development Team has no role in implementing the software in a live environment. This might be the case when that specific task is to be performed by the IT Operations Department. From a test manager’s perspective this caveat needs attention because it leads to non-addressed risks in the period between the deliverance of the Development Team versus the actual implementation in a live environment.

To overcome these potential trouble with the testing process different ideas have been introduced to deal with potential shortcomings. We will have a look at two of them: Agile Testing Quadrants and Test-Driven Development.

Agile Testing Quadrants

Developed by Brian Marick it places different tests in different quadrants based on their relevance (technology driven versus customer driven, supporting the team versus product critique). An advantage of this visual representation is that the Development Team is aware of the tests that support the quality of their products.

Test automation is an important aspect of trying to solve issues of reducing the testing time it will take from the resources of the Development Team or stakeholders to perform tests. We need to be aware though that test automation is not the ultimate cure for time related issues. Tests have to be coded, so it takes time to complete them before the test automation can facilitate the products that emerge in Quadrant 2. Test ware needs to be maintained as well. For example external factors like an organisation upgrading to a new version of a browser or a fixed security leak need to be taken in account when keeping the test ware up to date.

In Quadrant 4 tests like Performance, Load and Security Testing are positioned. These type of tests require special expertise that is not necessarily available by default within the Development Team. Also the same as for test automation is applicable: the test ware for Performance, Load and Security Tests needs to be initially produced and afterwards actively maintained in order to keep them relevant or simply working in future Sprint iterations/ increments . With Security Tests you might even ask if it is desirable to run these test from within the team due to their delicate nature and from a perspective of segregation of duties.

Test-driven development (TDD) has its origins in extreme programming, an other Agile method. With TDD the developer starts his work by writing the actual test before writing the software code for the product to be delivered. Although TDD has a lot to be said about that goes beyond the scope of the article the clear advantage is that it solves issues raised in previous paragraph. The creation and maintenance of test ware becomes an integral part of the work by the Development Team.

Conclusion
Scrum needs an organisational change of mentality to gain the best results. If only the IT software development is isolated and using Scrum, while other parts of the organisation (Marketing, Finance, Purchase, HR) do not participate, Scrum will be less relevant or even fail to work. But even within the IT department itself the integration of Development Team with the Operations organisation, becomes unavoidable and mandatory to tackle the problem of the possible gap between the Scrum product according to the Definition of Done and the ‘real’ live implementation date.

Test automation is necessary to keep control over the time aspects concerned with testing. But as test automation needs its own level of maintenance it is certainly not the answer to all challenges that the Development Team faces. Especially the tests in Quadrant 4 (Performance, Load, Security) require special expert knowledge that will not always be present by default from the Development Team. If that is not the case it will need to be planned with the required resources and might lead to more planning or product related issues that are difficult to match with the short cyclic iteration of the Sprints.

Apple Pay evolution to the blockchain

In its current format Apple Pay is depending on traditional banking and credit card institutions. This has not proven to be a guarantee for a fast world wide roll out of the service. There are tough negotiations involved both on the side of merchants as well as the financial institutions.

For example in The Netherlands there have been talks about rolling out for more than a year. Finally next month things should be ready for ‘prime time’. I have no idea what the final ‘service’ will look like in the Netherlands. It surely will be along the lines of scanning your bank debet card(s) and credit cards (hopefully all financial institutions will be involved) in order to enable transactions. The final seal of the transaction will be made through the Touch ID of Face Recognition when iPhone X will make its way to the community. There is the challenge to attract  merchants to accept Apple Pay. Judging from the acceptance in the USA where Apple pay already is active, another possible factor that the final service will be somewhat crippled from the start.

Suppose that in the near future Apple would be skipping the financial institutions and operate as a financial intermediair supported by a decentralised public ledger (blockchain). It would look something like this.

Blockchain, currency

Apple starts the blockchain with its own currency, opening the ‘to be’ blockchain to miners processing transactions and offering financial compensation delivering the computational power for transaction processing.

Consumer

The consumers are able to buy Apple’s own currency, exchanging fiat money for the Apple Currency (which I shall name ACX for convenience reasons). This process is from a consumer perspective similar to buying something from the App Store. Only the exchanged money does not land on Apple’s bank account but on the blockchain, so Apple has a function here as an exchange institution (from fiat money to ACX). As soon as the exchange transaction is processed (all made possible through an Apple device like an iPhone, iPad with a dedicated app) the ACX balance of the consumer is updated and visible from the app.  The consumer will now be able to spend his ACX balance online or through Apple Pay at physical stores.

Transaction with merchant

The merchant who is affiliated with Apple Pay is then capable of adding balance to his Apple business account also stored on the blockchain. The transaction between consumer and merchant takes place between two Apple devices for example an iPad on the merchant side and an iPhone on the consumer side. There must be a kind of handshake between the two devices, finalized by a ID on both merchant and consumer side. The consumer will be debited for the transaction, the merchant credited. Apple will allow the merchant as well as the consumer to exchange their ACX for fiat money (USD, EUR etc.). Transaction will processed by decentralised processes on the blockchain enabling miners to receive a fee for the transaction.

Conclusion

I am sure that solutions along the lines as described above will be made available in the near future. I can imagine that Apple choses its own blockchain, it has the financial power to do so. Not only by Apple but all the major big players (Google, Facebook, Microsoft etc.) wil start offering services along these lines. I use the example of Apple because they are probably already ahead of the bunch and already have an integrated ecosystem.

The implications are far stretching: Apple becomes a financial institution itself and largely independent from the traditional banking and credit card institutions. Therefor it may decide to start a specialised branch named Apple Pay. The underlying blockchain technology will be essential to guarantee the integrity and security of the ecosystem. I also understand such a scenario will have far reaching fiscal and financial consequences which I will not address here, but in general Apple’s dedicated branch will behave very much like a traditional financial institution from a fiscal and financial perspective.

The Hague, October 21th 2017