HR Magazine – Debt Crisis

Click here to view original web page at www.shrm.org

When the computer that ran the web-based applicant tracking system crashed, recruiting came to a screeching halt at a bank company with 24,000 employees in 13 states. The tracking system, which interfaced with the HR information system, remained out of commission for 48 hours while a bevy of database administrators, software developers and hardware experts at the company and the outsourcing partner scrambled to solve the problem.

Because the partner had built and maintained the system for several clients, “three companies were dead in the water. The clients were not happy,” recalls Aaron Callahan, PHR, who worked for the outsourcer at the time. “All of a sudden, the career portals where people searched for jobs and added information to their profiles went down. It was all hands on deck to find out what was going on. We had 15 to 20 people working round the clock.”

What caused the crash?

Technical debt. This software development concept uses the model of financial debt as an analogy for the long-term consequences of poor design decisions in architecture and of customization, shortcuts and work-arounds in the code base that accumulate over time. The technical debt in Callahan’s example accrued from layers of customization until the hardware and software supporting the system could not take it anymore, resulting in the analogical bankruptcy.

Vendors and end-users need to understand technical debt and the resources it consumes, and must have a plan—as with financial debt—to pay it off. This is true whether you use an outsourcer or software-as-a-service vendor, create and maintain your own HR software, or use internal information technology and HR information system resources to operate on-premises systems.

“If you think this is only of interest to techies, please think again,” says Naomi Bloom, a strategic HR technology consultant and managing director of Bloom & Wallace in Fort Myers, Fla. “What end-users don’t know about what lurks in the corners of their software definitely hurts them.”

Sludge in the Tank

Over time, software systems that are not properly maintained become more complex and their structures deteriorate. Little by little, technical debt accrues due to the way developers structure the architecture and write the software, how they maintain and update it, how they customize it, how they integrate it with other systems, and how vendors weave technologies together when they acquire applications from other vendors. And there’s no shortage of acquisitions these days.

Part of the problem is that enterprise software nearly always needs changes over time. For example, if HR software is designed to meet U.S. requirements but then needs to evolve to accommodate global needs, “the software may need reworking, and that can add to the technical debt,” explains James Holincheck, a managing vice president for human capital management and other enterprise applications at Gartner Inc., a technology research and advisory firm based in Stamford, Conn. “The technical debt may not be obvious until five years from now. You recognize the debt is there but do your best to minimize it. A younger vendor with a new product will naturally have less debt.”

Bloom says there’s more technical debt in commercial products now due to years of neglect by vendors. In an impassioned blog post last year, she wrote, “It’s the sludge at the bottom of your vendor’s software ‘tank,’ and it creates a considerable drag on vendor profitability, speed-to-market for new functionality, operational stability and error-free updates, etc. And buyers are paying for all of this with their licenses/subscriptions/maintenance, not to mention their manual work-arounds, customizations, add-ons, duplicate systems, disengaged workforces, impossible to change business rules, etc.”

Bloom acknowledges that the best vendors regularly reduce technical debt by redesigning the architecture, restructuring the code and rebuilding their products. “But many vendors don’t do nearly enough,” she wrote. “They prefer to invest in new features, new technologies, marketing and sales—in things buyers and investors can see. And who can blame them when many market influencers/analysts, let alone buyers, rarely mention or even understand what’s under the covers.”

This hurts customers, Bloom argues, because “the amount of technical debt lurking under the covers of your seemingly shiny new software has a major impact on whether or not you achieve the very benefits for which you’ve committed resources to this vendor and to the implementation of this software. The more debt, the less likely you are to get those benefits now and into the future.”

Tough to Avoid

It isn’t just vendors that accumulate technical debt. Although less HR software is developed and maintained within user organizations than in the past, integration work done internally or by outsourcing partners adds to debt, as does—the real culprit—customization.

Customization was at the heart of the debacle described by Callahan. The customization was requested by the customers and delivered by the outsourcing partner without much attention paid to the debt buildup. “We never made time to go back and clean things up,” he says.

‘There is a lot of technical debt to clean up related to mergers and acquisitions.’

Today, Callahan is the HR information system administrator for VyStar Credit Union in Jacksonville, Fla., with 20 branches and 1,093 employees. He reports to the senior vice president of HR. The credit union’s technology partner is Automatic Data Processing Inc. (ADP) in Roseland, N.J., a provider of payroll services and human capital management software and services.

At this point in his career, Callahan says he is “anti-customization” and a fan of software as a service. But even that model doesn’t do away with technical debt, he admits: It “does not necessarily eliminate technical debt if a vendor has acquired a suite of products or apps they sell as a holistic solution. These products were developed on different code bases. There is a lot of technical debt to clean up related to mergers and acquisitions.”

Callahan is encouraged that representatives for his vendor have stated an intention to reduce technical debt in their products. In a LinkedIn discussion on this topic in November 2011, Bloom cited ADP and Ultimate Software Group Inc., based in Weston, Fla., as vendors whose officials have made such public commitments. Other vendors are expected to follow suit.

Debt Reduction

Dovetail Software Inc. in Austin, Texas, develops software that helps manage enterprise support and service functions of all kinds, including HR. Steve Lynn, chief executive officer, is committed to tackling technical debt. He acknowledges that privately held Dovetail is cushioned somewhat from outside pressures that lead publicly held developers to step up schedules in ways that cause more technical debt.

“As a small company, we have to be sustainable,” Lynn says. “We have to make sure we are not going backward or sideways in three years. As a private company, we can do that. Even though there is a cost, we deal with technical debt now or we won’t be in business.”

Dovetail’s developers began to deal with technical debt in 2008, when they decided to rewrite the HR product. “We took a year to rewrite that product, keeping in mind the flexibility we wanted for the future,” Lynn says.

Chad Myers, Dovetail’s director of development, says technical debt occurs for three reasons:

• Intentional decisions, usually schedule-driven, to do things that will cause technical debt.
• Unintentional decisions, due to poor or lazy development practices or untrained developers.
• Forces outside developers’ control.Open communication between development staff and their business executives serves as an antidote to the first cause; installing best practices and training developers thwart the second, Lynn and Myers say. Good project management is imperative, Myers adds. Even the vendor’s business leaders contribute to the technical debt by imposing unrealistic demands on developers, he notes.Myers estimates that Dovetail budgets 30 percent to 40 percent of its resources in each development cycle to reducing technical debt created in earlier projects.A rule of thumb for both vendors and end-users is to allocate at least 10 percent of resources in a project such as product development, customization or integration to “pay down the debt,” advises Steve McConnell, CEO and chief software engineer at Construx Software, based in Bellevue, Wash. Construx offers consulting and training services for software development to Microsoft, Google and other clients.Payoff PlanJust as there are legitimate reasons to take on personal debt—a car, a house, college—there are legitimate reasons to take on technical debt, McConnell argues. As with personal debt, if you don’t have a plan to “pay down the principal” of technical debt, your system will eventually collapse under its weight. “The debt may not seem like much at the time, but the slow accumulation of debt over time becomes the problem,” McConnell says.

  Keep a paper trail of structural changes made during technology projects, adds Jeremy Ames, president of Gaucho Group Inc. in Bellingham, Mass., an HR information systems consulting firm. Ames serves as a board director of the International Association for Human Resource Information Management Inc. “Document everything you do during implementation and customizing so it can be referenced,” he urges. “Version control is not as common as you would like it to be in the world of consultants and vendors who need to make changes.

  “There needs to be checks and balances with the development team.”

  Since most HR software now comes from vendors, include technical debt as an aspect of vetting products and vendors during purchasing decisions.

  Callahan urges adopters to create their own scripted scenarios and put the software through its paces. Vendor demos “will show the system to be perfect and flawless,” he says. “Document your scenarios and apply those to their solution. That is one good way to flesh out if there is lurking technical debt.”

  In her blog, Bloom provides links to “killer scenarios” for this purpose.

  Quality Control

  HR tech managers agree that technical debt will not go away. “I’ve been on three calls today with companies that are making decisions about avoiding it, making decisions based on it or suffering from it,” Ames says. “Every decision that you make on a systems project either goes on one side of the ledger or the other. Are you creating technical debt from that decision or reducing it?

  The author is technology contributing editor for HR Magazine and is based in Silicon Valley.