Why backend services become difficult to change

Maintainability is the cost of making a safe change

A maintainable service is not necessarily small, new or written in the latest language. It is a service whose behaviour can be located, changed, verified and released without requiring one person to reconstruct its entire history.

Line count and code style are weak proxies. A five-hundred-line service can be expensive to change when it has an unreproducible build, undocumented consumers and no useful production evidence. A much larger service can remain workable when its boundaries and feedback loops are clear.

Assess maintainability through the path of a real change. How long does it take to find the responsible code, understand the affected contracts, run meaningful tests, deploy the result and determine whether it worked?

• Can a new checkout be built and run?
• Can the relevant behaviour be found without reading the whole repository?
• Does the change remain inside a comprehensible boundary?
• Do tests fail for the mistakes the change could introduce?
• Can the release be observed and reversed?
• Is the reason for an unusual constraint available to the next engineer?

Follow one apparently small change

Consider adding displayName to an account response. The visible request sounds small, but the value may pass through Play routing, JSON validation, an application command, a domain model, MongoDB mapping and an external response contract.

The change becomes risky when the same case class owns all of those representations. A transport field made optional for compatibility can spread Option through domain code where a name is required. Renaming a persistence field can make existing documents unreadable. A derived JSON writer can change the public API because an internal field was renamed.

Draw the path before editing it. The number of layers is not the problem; the difficulty comes from boundaries whose contracts and owners cannot be distinguished.

HTTP JSON
   ↓ Play Reads
OpenAccountRequest
   ↓ validated translation
OpenAccount command
   ↓ application service
Account domain value
   ↓ persistence mapping
MongoDB document
   ↓ presenter / Writes
AccountResponse JSON

Ask which representation is allowed to change at each arrow.

Unclear boundaries spread the blast radius

Controllers should own HTTP concerns, application services should express application behaviour and repositories should own persistence translation. These are responsibilities, not mandatory folder names. A small service may combine some code while keeping the contracts distinct.

Difficulty appears when a service returns JsValue because only the current controller calls it, or when a repository exposes MongoDB query documents through application code. Callers then depend on transport or storage shapes without admitting it in their signatures.

Keep meaningful values typed until the boundary that owns serialization. This makes an HTTP rename, storage migration or new adapter a local translation problem rather than a change to every caller.

// Transport and persistence concerns are mixed:
def find(id: String): Future[JsValue] =
  collection.find(Json.obj("_id" -> id))
    .map(document => Json.toJson(document))

// The application contract remains typed:
def find(id: AccountId): Future[Either[FindError, Account]] =
  repository.find(id).map {
    case Some(account) => Right(account)
    case None          => Left(FindError.NotFound(id))
  }

// The controller maps Account to AccountResponse JSON.

Hidden effects make signatures misleading

A method named calculateSummary is difficult to reason about if it also writes a document, calls another service and records an audit event. A reader cannot predict failure, latency or retry behaviour from the apparent calculation.

Expose asynchronous work and expected failure in the return type where practical. Keep pure calculations as ordinary functions, then compose persistence and external calls in an application operation whose name and signature acknowledge those effects.

Be especially careful with recovery. Converting every failed Future into None hides an unavailable database as missing data. Retrying an undocumented write can duplicate a side effect. A fallback can return stale information without telling the caller.

def rename(
  account: Account,
  name: AccountName
): Either[RenameError, Account] =
  Either.cond(!account.isClosed, account.copy(name = name), RenameError.Closed)

def renameAccount(
  id: AccountId,
  name: AccountName
): Future[Either[RenameError, Account]] =
  repository.find(id).flatMap {
    case None => Future.successful(Left(RenameError.NotFound(id)))
    case Some(account) =>
      rename(account, name).traverse(updated => repository.save(updated).map(_ => updated))
  }

Asynchronous code carries operational assumptions

A Future communicates that a result arrives later, but it does not reveal whether the underlying operation blocks, which ExecutionContext runs it or whether the work can be cancelled. Wrapping synchronous database or filesystem work in Future does not make that work non-blocking.

Long-lived Play applications can become fragile when blocking calls share the default dispatcher with request processing. A slow dependency then reduces capacity for unrelated endpoints. Execution contexts, timeouts and resource limits are part of the service design.

Concurrency changes require the same scrutiny. Replacing sequential calls with Future.sequence may reduce latency for ten bounded items and overwhelm a dependency for ten thousand. The service needs an explicit capacity policy, not only asynchronous syntax.

Data compatibility outlives the current code

Stored documents and external consumers survive individual application releases. A case-class field rename may compile cleanly while changing derived Play JSON or MongoDB serialization. A new validator may reject requests that existing clients still send.

Review representative stored data as well as Scala types. Define how old documents are read, whether new code writes only the current representation and when compatibility logic can be removed. Make defaults deliberate; silently manufacturing a value can hide a migration gap.

Indexes and query shapes belong in the same review. A functionally correct query can become the most expensive part of the change at production data volume, while a new unique index can fail because old data already violates the proposed constraint.

implicit val accountReads: Reads[AccountRecord] = (
  (__ \ "accountId").read[String] and
  ((__ \ "displayName").read[String] orElse
    (__ \ "name").read[String])
)(AccountRecord.apply _)

implicit val accountWrites: OWrites[AccountRecord] =
  Json.writes[AccountRecord]

// Compatibility is visible, testable and removable later.

A large test suite can still provide weak confidence

Test count and coverage do not show whether the important behaviour is protected. A suite can contain hundreds of mock-heavy unit tests while never exercising JSON codecs, dependency injection, database queries or failure translation.

A useful unit test protects a decision and tolerates an internal refactor. An integration test exercises a real boundary whose semantics matter. If a repository test mocks the MongoDB collection, it proves how the mock was configured rather than whether the query can round-trip the production document.

Read failure output as part of maintainability. Tests that are slow, order-dependent or vague increase the cost of every change. Flakiness teaches engineers to rerun evidence until it agrees with them.

• Which business rules have fast value-based tests?
• Which transport and persistence contracts are exercised for real?
• Do asynchronous assertions return or await their Future?
• Can tests run independently with controlled data?
• Does a failure identify the responsible boundary?
• Can safe refactoring occur without rewriting interaction scripts?

The build is part of the service architecture

A readable codebase is still difficult to maintain when only one warm development machine can build it. Record the Scala and JDK versions, sbt version, compiler plugins, private dependencies and configuration required by a clean checkout.

Long feedback loops change engineering behaviour. When compilation or tests take too long, developers batch changes, avoid running checks and receive failures after the relevant context has gone. Separate fast local evidence from slower integration work without making the latter optional.

Dependency warnings and temporary overrides should have an explanation. An unexplained exclusion can be the only thing preventing an incompatible transitive version; deleting it during routine cleanup can create a runtime failure far from the edit.

show scalaVersion
show javaHome
show sbtVersion
plugins
show Compile / scalacOptions
dependencyTree
evicted
clean
Test / test

// Run from a clean checkout with the same JDK used by CI.

Production feedback determines diagnosis cost

A failed request reported only as “internal error” forces an engineer to infer which boundary failed. Structured logs should carry safe identifiers and operation context, while preserving the original cause for unexpected failures.

Metrics and dashboards should answer whether the changed path is used, slow or failing. Correlation between a request, downstream call and database operation shortens investigation far more than adding unrelated log lines everywhere.

Observability must reflect expected rejection as well as operational failure. A validation error, missing account and unavailable database are different events. Treating all three as exceptions creates noisy alerts; treating all three as ordinary absence hides incidents.

service.rename(id, name).map {
  case Right(account) => Ok(Json.toJson(AccountResponse.from(account)))
  case Left(RenameError.NotFound(_)) => NotFound
  case Left(RenameError.Closed) => UnprocessableEntity
}.recover {
  case error: RepositoryUnavailable =>
    logger.error(s"Rename failed for account ${id.value}", error)
    ServiceUnavailable
}

Ownership gaps become technical constraints

A service is difficult to change when nobody can explain which consumers rely on an endpoint, why a workaround exists or who can decide ambiguous domain behaviour. Engineers respond by preserving everything, adding another branch and avoiding deletion.

Useful documentation records decisions, contracts, operational procedures and the reason behind surprising constraints. It does not need to narrate every class. Keep information close to the code or workflow it governs and update it when the decision changes.

Ownership does not mean one permanent expert. It means there is a reachable person or team responsible for decisions, and knowledge is visible enough that another engineer can take part without an oral history.

Improve the path of the next real change

Do not begin with a rewrite or a repository-wide cleanup. Choose a real product change, recurring failure or dependency constraint and trace its complete path. That provides a reason to improve the boundary currently creating cost.

Protect existing behaviour at the narrowest useful level. Separate compatibility work, refactoring and new behaviour where possible. Make one responsibility clearer, run the relevant integration evidence and release a change small enough to observe and reverse.

Record what remains difficult. Maintainability improves through repeated reductions in change risk, not through declaring the architecture clean.

• Choose one valuable change or recurring failure
• Trace input, decisions, effects and externally visible output
• Identify the boundary creating the most uncertainty
• Protect the behaviour that must remain
• Refactor and change behaviour in reviewable steps
• Deploy with an observable success and rollback condition
• Leave the next constraint documented and prioritised

The service should answer the next engineer back

A maintainable service is not one that never needs explanation. It gives the next engineer enough evidence to understand a change, enough boundaries to contain it and enough feedback to know whether it worked.

The strongest signals are practical: a reproducible build, types that preserve meaning, tests at the boundaries that matter, compatible data changes, direct ownership and production behaviour that can be observed.

When those signals are weak, even a small service becomes expensive. When they are strong, a mature backend can continue moving without pretending its history and constraints do not exist.