Student Management on the AWS Cloud

James Bromberger Posted 23 January 2020

In 2018, a government department in Australia was looking for options to deploy its new student management system, then in active development through a third-party software vendor.

The Department brought in a small Modis team to design and build a best-practice deployment pipeline and infrastructure on Amazon Web Services (AWS).

The Challenge

The AWS infrastructure needed to meet several key requirements:

  • Be able to scale dynamically to meet peak performance periods at specific times of the day, that might be 10,000+ logins in a 30-minute window.
  • Since the roll out of the application to all the Department’s 800+ locations would be a gradual process, the infrastructure had to start small and be able to grow in a cost-effective way.
  • It had to seamlessly integrate with several on-premise repositories and services for regulatory requirements.
  • Given the nature of the information being processed and stored, end-to-end encryption of all data in flight and at rest was mandatory.

Also, the third-party software was still in active development, with meant the deployable artefacts were in a constant and unpredictable state of flux. Still, this software had to be worked into an automated deployment process.

The Solution

The Department already had a significant investment in on-premise hardware and software licences, so any cloud infrastructure had to operate in a hybrid environment. For the solution to work several key on-premise repositories and resources had to be accessible from the Cloud and vice-versa. The Modis team used a relatively new feature of Route 53 called Route 53 Endpoint Resolvers to integrate the on-premise private DNS namespace, with the Virtual Private Cloud environment. From the point of view of servers in the Cloud, DNS resolution was seamless.

Deploying the software artefacts handed over by the vendor proved to be a complex, multi-step process involving sequenced database updates, provisioning multiple Microsoft IIS servers containing multiple applications, and procedural elements such as clearing the Elasticache cache at specific times.

To make the deployment to AWS quick and reliable with these constraints, automation was critical. All AWS resources were created by CloudFormation templates, broken down by function or lifecycle, and parameterised so they could be orchestrated and executed in many ways as required. Detailed configuration of the Microsoft Windows application servers or complex behaviour was handled by PowerShell scripts, modularised to share common behaviour, and parametrised for maximum configurability.

Given the changing nature of the deployable artefacts during development, regular lessons-learned sessions were held so that feedback could be provided to the third-party vendor on ways to make the deployment process even faster and more efficient.

All servers were placed into auto-scaling groups (ASG) behind application load balancers (ALB) to enable scaling in the face of demand. This allowed the EC2 fleet to grow and shrink as required and also ensured that the Department was minimising what it was paying for. Certain peak periods in the morning and afternoon were well know, so a feature of ASG was used to schedule the scale-up to ensure resources were pre-warmed to meet this demand.

AWS Systems Manager was used extensively throughout the project. All variable configuration items — including secrets and confidential information, such as usernames, passwords, and other tokens — were abstracted and stored in an encrypted format in a Parameter Store hierarchy. This ensured no sensitive information was ever committed to the source code repository. Systems Manager Maintenance Window and Automation was also used to ensure Windows and Linux security updates were automatically applied across the EC2 instance fleet.

Given the highly confidential nature of the data being processed and stored, end-to-end encryption of all data in-flight and at rest was mandatory. AWS Certificate Manager was used to provision and manage all CloudFront and ALB certificates with the latest cipher suites. Self-signed certificates were then used for the hop between the ALB and EC2 instances themselves. All S3 buckets had versioning, encryption, and public access-denial enabled by default and built into the associated CloudFormation templates.

Continuing the defence-in-depth strategy, AWS Web Application Firewall was used to implement geo-restriction to known access points, along with a series of Open Web Application Security Project (OWASP) proactive controls such as cross-site scripting and SQL injection.

To maintain the security posture of the AWS infrastructure itself, CloudWatch metrics, alarms and events were used to monitor key resources such as the root account, security groups, S3, and IAM permissions. Any out-of-band changes triggered a real-time message to an SNS topic, which forwarded a email to the DevOps team to investigate.

Managed security services such as Trusted Advisor and Guard Duty were activated as a matter of course. Again, CloudWatch events were used to flag any deviations from the known steady-state.

To support the eventual transition to production and support, all application server logs and key operating system logs for the EC2 fleet were pushed to CloudWatch Logs along with VPC flow logs and CloudTrail logs. An Elasticsearch cluster then ingested this log stream. Technical and customer support teams could then use Kibana to search and visualise the data for support purposes.

Outcomes and Benefits

The resulting AWS infrastructure proved it was possible to take a complex and changing software application and work it into an automated deployment, while still meeting key goals for scalability, cost optimisation, and security.

Find out how Modis can provide you with innovative AWS cloud based solutions and servicesModis has been an AWS Advanced Tier Partner since 2014. Modis' AWS Cloud Consulting services encompasses fundamentals of cyber security, fault tolerant digital system architecture, modernisation, traditional virtual machine or through to modern Serverless approaches, commercial off-the-shelf software operation to bespoke software development, delivered with high throughput, repeatable DevOps approaches to operations. With over half a decade of running critical authoritative government data sets that affects the lives of millions of citizens and the economies of the state, Modis has one of the most mature, experienced and recognised consulting service providers in the world. More importantly, we like to work very closely with our customers, not providing something to purchase, but taking a deep understanding of their business, and providing the recommendations and implementations to ensure a modern, efficient, reliable and secure environment for digital business systems.Contact us
We operate around the world. Would you like to find out more about your local office?Find out about Modis