Choosing web dev tools in a constantly changing tech world

Published Feb 23, 2021, 9:32 AM
Written by Marcus Nyeholt

The reality of web development is that there are hundreds of tools that are suitable for the job now but which might not be as useful when requirements change – which they’re likely to do next week. 

Some Background

Though I was somewhat late to the web development scene – remember back to 2004 when IE was already at version 6? – there were already several open source CMSes around that made it easier to get content online from a local system.

Things have changed, but some things have stayed the same. While open source tools definitely have their advantages, the problem is now more about which to choose, since there are so many available. It can also be hard to develop expertise in specific tools and languages while also keeping current with new developments.

We manage to do it because our Symbiote team lets us spread the effort, digging into specialised aspects of the foundational tools that allow us to make websites and manage the inter-related databases and systems that make them function. At the same time, we’re also able to keep our eyes open for opportunities to try out and implement new languages and tools, which keeps us nimble and flexible (and makes sure that we’re having fun).

Our philosophy has always been to choose the right tool for the job. 

The reality of web development is that there are hundreds of tools that are suitable for the job now but which might not be as useful when requirements change – which they’re likely to do next week. 

Our decision making sometimes involves educated guesses, trying things out and being flexible in adapting new technologies or techniques into existing frameworks. 

Here’s a deep-dive into our thinking and the way we manage our tech, so we keep up with emerging technologies while giving our customers the stability they expect. 

How we choose the technology we use at Symbiote

While the specific tech pieces at Symbiote have changed and evolved over time, these are the core principles we keep in mind when making our decisions:

  • Open – issue resolution often means looking under the covers of various systems to know how things work. Open source has been critical for us. Aside from access to the code, the community surrounding open source is always accessible. It’s also satisfying to share things we’ve worked hard on, knowing it’ll make someone else’s work-life easier.
  • Maintainable – we work with customers for years; it’s essential to us that their underlying systems are built for the long haul. That helps them and it certainly helps us to make updates and improvements.
  • Traceable – any change can be traced back from whatever a customer experiences on the front end to the code that produced it.
  • Performant – we can scale projects with little effort if needed.
  • Consistent – developers can work on a system that closely resembles the production environment, and have confidence that the same actions will have the same outcome.

Let’s dig into these:

Open

Our technology set is made up of primarily open source tools that sit atop a proprietary infrastructure platform in Amazon Web Services (AWS).

A key driver of our original content management choice was a BSD-style license that gives us true freedom. This flows into other open source projects throughout:

  • Silverstripe CMS
  • Git and Gitlab
  • React
  • Terraform and Packer
  • Docker
  • Cypress

Maintainable

PHP allows developers to build some truly remarkable code (without some of the disastrous side effects and monstrous code it can be known for) so long as they use well-designed and structured frameworks to share, modify and extend that code.

We ensure that all of our code is easily maintained by using Silverstripe CMS with additional frameworks and libraries like Slim Framework and React. These frameworks come with useful ecosystems and tools like composer and yarn that manage dependencies maintenance via automated version checks and dependency resolution.

Gitlab CI lets us turn a code update into a deployable package and get it out to live environments in minutes. And Terraform gives us the power to rapidly update all aspects of the infrastructure environment.

Traceable

It’s critical to understand why things are behaving the way they do when an issue arises. Tools like New Relic, LogzIO and Cloudwatch allow us to trace issues back to a cause.

Traceability is about more than just tooling. To make automated browser testing simpler, our developers tag important elements in the output, making testing and component identification simpler when issues do arise. Over the years we’ve done a lot of work with Selenium based solutions for browser testing. We’re currently working a lot with Katalon as a shared test development tool. Now that Cypress.io has wider browser support, we’re looking to work more in that space too.

Tools like Gitlab let us trace back a deployed environment state to the commit from which that entire server was deployed from. This gives us an end-to-end view of all things in play for any issue that arises.

Performant

‘Will our solution scale while being re-useable in different environments?’

That’s the question we discuss as a team to make sure that we choose the right techniques and hardware to manage websites, especially those that have high traffic flows.

While some developers will throw hardware at sites that manage thousands of dynamic requests each minute, this can get expensive and it isn’t necessarily effective. It also adds to the ongoing maintenance effort.

To make sure our websites are responsive and perform now and in the future, we focus on caching and APIs, using the built-in capabilities of Silverstripe’s CMS and edge services at Cloudflare that serve content before requests ever get to the application.

When it comes to providing a shared cache for distributed deployments, we go for AWS-provided services like Redis and DynamoDB and use flexible policies whenever we think that temporarily adding hardware will solve specific issues.

Lightweight frameworks like Slim and AWS Lambda allow us to deploy APIs that serve thousands of dynamic requests per minute, and frontend technologies like React and Redux help with building out logic-heavy user interfaces.

Consistency

Using Docker lets us replicate, as closely as possible, the configuration of the production environments our customers are running. 

Consistency comes from using the same configuration in our continuous integration environment, so we’re confident that our automated test and build will be as close as possible to what the end user will see.

Here’s a summary of the tech we use at Symbiote

Like many organisations, we started with bespoke third party hosting services, a self-hosted subversion system for storing code, and a bunch of scripts for moving code around. As we’ve matured, our team has grown, and our customers’ needs for both rapid change coupled with reliability has provided interesting technical problems. So while our core content management system has the same name, both it and the technologies surrounding it have improved over time.

We’ve moved from jQuery to React, bespoke self-managed hosting to AWS, scripts for build packages to GitlabCI, and shell scripts for deployment to Terraform. 

 

Component

Role

What

Web stack

Handling web requests from users, delivering content back

Linux, Apache, MySQL, PHP7

Web Content Management Systems (CMS)

The application that allows content authors to edit and manage content items, text and images

Silverstripe CMS

PyroCMS

Frameworks

Code libraries used for building sites and applications, utilising content outside of the web CMS 

Slim Framework

React

Node

Lambda

Source control

Managing versions of source code and related assets 

Gitlab

Github

Developer environments

Systems that developers use to develop code

Docker

Infrastructure

The (virtual) hardware that applications are installed on

Amazon Web Services (AWS)  

Continuous integration (CI)

Ensuring that code components can be ‘built’ into deployable packages

Gitlab CI 

Deployment

A system that takes code from a source repository and delivers it to remote environments, ie the live infrastructure

Ansible

Packer

Terraform

Content Delivery Network (CDN) services

Sitting between the user’s browser and application servers, a CDN provides security and performance assistance for the underlying applications

Cloudflare

Testing frameworks

Confirm that code works as expected, and that sites run as required

PHPUnit

Katalon

Codeception

Monitoring

The systems used for monitoring the live environments for performance and stability

New Relic

LogzIO

OpsGenie

xhprof

AlienVault

And that’s where we’re at

Symbiote’s grown to the point where we’re now working on big projects – you know, ones that affect millions of end users. We still use the technology stack we’ve developed over the years on every project we develop, but we structure it so we’re only using the components that are the best fit for each project, with our developers having the flexibility to choose how things are put together.

You can find the open source material we share on GitHub (https://github.com/symbiote/).

I hope you’ve found it interesting and useful to see how we build and use our web development technology at Symbiote. Feel free to get in touch if you’re a web developer who likes the idea of working with a well-understood core of technologies and continuing to learn and add in new tools to solve interesting problems.