AVL Release Notes
v6.3.1
New Features
Universal Lab Launcher
This feature allows for easy embedding of AVL labs into any web page. For more information check out the documentation.
v5.7.1
Bug Fixes
Correct Launching Error Results
In some cases, attempting to launch a non-existent project, or trying to launch a project without a user email would give an incorrect error response. This is now giving the correct error response.
v5.7
New Features
Deployment Event Logs
Deployment Event logs are written and available to download from: <yoursite>/reporting/raw_history/csv
The events recorded are:
DEPLOYMENT_STARTED DEPLOYMENT_FAILED DEPLOYMENT_RESTARTED DEPLOYMENT_READY
Bug Fixes
Allow Lab Admins to change port settings of existing Labs
Previously the admin user could not update the port settings once a lab had been launched, which caused the Appsembler team to have to update the settings with a workaround. Now if there are no running labs tied to these image settings, the ports can be modified.
Clean up containers after failed launch
This fix consists of two parts. The first part fixes the issue where labs were not always cleaned up well after they have failed to launch. The second part is to run a script to periodically clean up these containers to handle any failures from before the fix was applied. The script continuing to run is a precautionary measure in the case of other scenarios where we don't clean up properly, though as yet we don’t know of any bugs in lab cleanup that have not been resolved.
Improvements
Handle failed lab launches more gracefully
In some cases, the response from the lab launch was not handled well, resulting in the learner seeing an opaque error message or no message at all. Now the learner will always see a message and the message will be informative.
v5.6.0
This version was not released.
v5.5.0
New Features
No significant new features were added in this release
Bug Fixes
No bug fixes were included in this release
Improvements
Fix and expand event logging
Throughout the lab lifecycle, we log events, and we have added two new event types: when a deployment fails and when a deployment is ready. Along with the existing events, for deployment started and deployment deleted, this can help us and you more fully to understand why a deployment was removed—simply because it came to the end of life, or because something went wrong in the launch process.
Handle timeouts better
Occasionally, a request for a lab will timeout in our application (rather than by us hitting the threshold for ping_retry). When this happens, we now respond more gracefully to the learner and log more information for us to debug and improve.
v5.4.1
New Features
No significant new features were added in this release
Bug Fixes
Use freed reserved host ports
The fix in 5.3.0 was incomplete and in certain cases where containers allocate a lot of ports, the ports were not reused correctly, this is resolved with this patch.
Improvements
No other improvements are included with this release.
v5.4.0
New Features
No significant new features were added in this release
Bug Fixes
Ensure the User Containers tab of the AVL dashboard is always available.
Instead of the User Containers tab displaying an error, it will show the unhealthy labs, but grey them out.
Note: Recently-launched labs may display as “unhealthy” if the dashboard was loaded as the lab was starting before it has fully launched.
Improvements
Improve cleanup up unhealthy labs
As part of the work above, changes were made that unhealthy labs get cleaned up correctly.
v5.3.0
New Features
No significant new features were added in this release
Bug Fixes
Use freed reserved host ports
Recently we’ve run into an issue with “reserved ports” in AVL. Each lab needs a number of ports open on the host machines so that the learner can access them. The limit was 20,000 total ports, over the lifespan of the cluster, and some customers hit this limit recently. With v5.3, these ports are now reused, which will also eliminate the symptoms of the port database table filling up (excessive logging and disk issues).
Add correct class name error
When an admin user tried to change the name of an event's class, the error wasn't displaying properly.
Fix the clickable area of the "Add port..." button in the image settings widget
The button for adding a port was sometimes obscured.
Correctly display error when changing Event Class name
When an admin user tried to change the name of an event's class, the error wasn't displaying properly.
Improvements
Add AVL_DEPLOY_ID to container env variables
In a lab, access the lab's deploy_id via the AVL_DEPLOY_ID env var.
Update NGINX buffer config
A limitation in the web proxy was identified with one customer, that affected requests with a large header size not being handled correctly. Note this is an optional deployment on an AVL cluster and will be enabled as required.
Notify if deleting an Image will also remove an Admin Container
In some cases, deleting an image would delete an admin container without first notifying the admin. We now stop before deleting to confirm
v5.2.0
We updated a key package that handles queuing lab launching and cleanup tasks. This will both give us further insight into why labs don’t always launch and improve the effectiveness of the launching process.
v5.0.7
This release contained internal improvements only.
v5.0.6
New Features
Display paired container logs
Added paired container's logs tab for user containers.
Improve Container Runtime Selection
Containers can now have their runtime set with units of a second, rather than days. The minimum run time is 1 hour and the maximum is 60 days.
Add Memory Allocation to reporting
The raw_history/csv output was not providing the full info for the images. With this change, we are able to see the RAM allocation for both, primary and paired images, which would allow us to estimate capacity at all times.
Bug Fixes
Properly set X-Forwarded-Proto header for container requests
HTTPS requests between students and labs were broken under certain circumstances. The current flow goes:
- A student initiates an HTTPS connection by clicking on a URL
- The request hits Nginx and terminates the TLS connection
- The resulting HTTP connection is proxied to the indicated Student Container and port
- The response is returned to the student's browser
In some cases, the application running on the student container would then send absolute links in the page content, including the protocol that it received the request on, incorrectly as HTTP. Updating the X-Forwarded-Proto header fixes this problem and ensures the protocol for future connections matches that of the connection used for the initial page load.
Make sure Event slugs are unique
Fixed a bug that allowed non-unique URL slugs to be generated from similarly named Events
Improvements
Update default image setting to use https
Update the default HTML widget code in image settings to use https rather than http in the container access URL. This only affects newly created image settings.
v5.0.5
New Features
Added token authentication to Projects endpoint
This allows for programmatic access to the list of AVL projects in a cluster
Serialized http_content field content in its serializer
Ensure that the API for launching containers returns the HTML snippet correctly in the JSON response
Added start and end dates to query string for /reporting/raw_history/csv endpoint
This allows for selecting a range of data for the reporting API endpoint to return
Bug Fixes
Made the Image Settings icon visible when the name is too long
In cases where an Image Settings had a long name, the button to edit the settings was not displayed in the AVL interface, this corrects that bug
Fixed environment variables in paired containers
This fixes a regression in v5.0.0 where the AVL environment variables were incorrectly set in Paired containers. This did not affect Primary containers.
Fixed footer logo on event pages
This fixes a bug on Event Student Dashboards where the Appsembler Logo overlapped the page content and prevented clicking the lab links.
Improvements
Implement correct deployment API response codes
The deployment API now returns more accurate HTTP response codes for success states, error states remain unchanged
Reload Nginx after renewing Lets Encrypt SSL certificate
Ensure that when the SSL certificate on an AVL cluster is refreshed, Nginx is reloaded to correctly serve the new certificate.
v5.0.4
New Features
Modify project's remaining deployments
We manage a project's remaining deployments differently than pre-v5, now that we have container cleanup on expiration. The number of remaining deployments is now calculated from the number of active deployments (those that are "running" or "stopped") rather than being calculated by summing the total number of historical deployments. So, if a project is configured to allow ten deployments, then launches five, then one expires, the project will have six (not five) deployments remaining.
This new behaviour was requested by several of our customers in the past. But please take note if you’ve been using remaining deployments as a means to cap the total number of deployments ever created – this behaviour has changed.
Bug Fixes
Fix Certbot across customer deployments
We made fixes to improve and standardize our use of Certbot across customer deployments. Certbot is part of the Let’s Encrypt ecosystem, and we use it to get and refresh SSL certificates for AVL.
This fix increases system stability and improves maintenance, but is informational – there should be no admin- or learner-facing change.
Fix race condition around new container creation
Fixed a race condition that allowed two containers to be spun up for the same Learner and Project. This caused problems in other parts of the codebase that only expect a single LearnerDeployment to be returned for a given (Learner, Project) pair.
This was one of the key bug fixes that we identified in v5.0.0 and prevented us from rolling out v5 more broadly.
Include all timezones in the event creation modal
A few users had noted missing timezones in the Event creation popup.
We were manually limiting the time zones that were displayed in that dropdown. We have now restored the missing timezones.
Restart stopped container with a call to /isc/newdeploy
We fixed a bug that prevented stopped containers from being restarted on subsequent calls to the /isc/newdeploy endpoint. The stopped container is now restarted, and then the container’s information is returned.
Improvements
Increased timeout for container start
For some images, the containers created from the image take a long time to spin up. In these cases, AVL was hitting a timeout before the request was completed.
This PR increases the timeout on the container creation request from 60 seconds to 90 seconds. The real solution lies in the software itself – speedup time needs to decrease. The lab author needs to speed up the time it takes the container to start. However, this should give extra time.
Performance improvements for learner_deployments endpoint
For AVL instances with a large number of learner deployments, accessing the learner deployments endpoint could lead to long load times, and repeated calls could overwhelm the main node. The fix involved adding date, email, event_name, and event_class_name filters to the /isc/dashboard/learner_deploymentendpoint.
v5.0.0
v5 is a refactor of our code, which will allow us to more quickly add new features and fix bugs.
New Features
Option to auto-delete labs at expiration
When the lab hits the "expiration date," it is now possible to take the (more reasonable) path: the lab is deleted rather than merely stopped. This is an optional feature. Unless we've talked with you, your labs will not be deleted, so plan to continue deleting manually. Let us know if you want your labs to be automatically deleted. (Note: this is a site-wide flag, so every lab will be deleted at the expiration date. Soon we hope to allow a more granular option, e.g., auto-deleting per-project.)
Container name and URL changes
During the deployment of v5, we will change the name of each lab. All labs will now use learner___<deployID> instead of user___<deployID>. The URL of each lab will also change, so you may encounter a learner here and there with stale links (for example, if a learner has saved the link as a bookmark). If you encounter an issue with a dead link, please ask these learners to return to the course page and click the (re)launch button for the lab. That button will return the new URL (but of course, it will be the same lab and lab content).
This change in URLs comes about because we are migrating all labs in the database. For a long time, we've had two types of labs in the database: one for live events and another for project / on-demand containers. Combining these into one table significantly reduces the complexity of our application and, as noted above, will allow us to complete features and bug fixes more efficiently and effectively. URLs work slightly differently with the learner___ structure, to which we're moving, so each migrated lab needs to align with this format. Hence, each lab URL must change.
Failed containers are deleted
If a container fails to launch, it will be automatically deleted. We consider a container "failed" when any of the steps do not work, including the final step where we check to see if your processes are running in the container. We realize that this may not be ideal in some cases; e.g., if you need to check the logs from the failed container or if you take other actions even when the container fails to launch. In v5.0.4, the next release after this one, we intend to improve this. However, if your processes are broken by not having failed containers in the admin UI, you may want to wait on getting v5. Please let us know if this is the case for you.
