Best practice forum (Archived)

Serious bug in 2.4.10 and 2.4.11

 
Simon Coggins
Serious bug in 2.4.10 and 2.4.11
by Simon Coggins - Friday, 11 October 2013, 11:07 AM
Group Totara

I am reposting details of this issue here so all subscribers can access it and reply:

As mentioned in the release notes forum we have discovered a bug which can lead to data loss in certain situations. The bug causes two issues:

1. Saving the program assignments tab can result in all program completion dates being deleted for that program.

2. The display of dates in other parts of the site can appear in UK English date format instead of the language's normal format.

We will have the fix ready for the 2.4.12 release which will be out on Tuesday 15th October.

In the mean time here is a bit more detail on who is affected and what you should do:

Affected versions:

  • Totara versions 2.4.10 and 2.4.11 only.
  • Sites which have one or more downloaded language packs installed only. Sites that only use the core English language are unaffected.
  • The bug above only occurs when users are making any changes to program assignments page when the user's language is not "English (en)". Users using "English (en_us)" are affected.

Mitigation actions:

  1. Avoid upgrading any site from 2.4.10 or below until 2.4.12 is available.
  2. For any site that has already been upgrade to 2.4.10 or 2.4.11, avoid saving the program "Assignments" tab until it has been upgraded to a fixed version. This will prevent any further data loss.
  3. For sites which have been affected by data loss, you may want to download a copy of the site logs filtered by 'update' actions for all days since the upgrade was done. This is only necessary if your site logs are purged after a certain period as we should be able to use the log data to figure out which programs are affected.

Finally if you absolutely can't wait until the fix is available to update your completion dates there is one other workaround:

  1. Login to the system as an admin user
  2. Switch the user's language to 'English (en)' (NOT 'English (en_us)' or any other variant).
  3. Update the program assignment settings. Everything should update as expected.
  4. Switch your language back but don't save the "assignments" page after that until the fix has been applied.

I will post again with more details about how this problem came about once we've got the fix out.

Simon

Simon Coggins
Re: Serious bug in 2.4.10 and 2.4.11
by Simon Coggins - Tuesday, 15 October 2013, 2:17 AM
Group Totara

This bug has now been fixed in today's release of 2.4.12. We urge you do upgrade any sites running 2.4.10 or 2.4.11 as soon as possible.

Simon

Simon Coggins
Re: Serious bug in 2.4.10 and 2.4.11
by Simon Coggins - Tuesday, 15 October 2013, 2:56 PM
Group Totara

Now that this bug is resolved we are investigating how it got through our QA processes and how we can prevent this from happening in future. We are also providing some more detail on how you can tell if you are affected.

Please accept my apologies if you were affected by this issue, people comment below if you have any additional questions or concerns.

Simon

Actions for customers affected by this bug

Steps to take

1. Avoid saving the program "assignments" tab for any site running 2.4.10 or 2.4.11
2. Upgrade any sites on 2.4.10 or 2.4.11 to 2.4.12 as soon as possible
3. For any older sites being upgraded skip the 2.4.10 and 2.4.11 upgrades and go straight to a more recent version (2.4.12 or greater)
4. Once the fix has been applied re-enter completion dates for any programs that have lost data. See below for how to identify affected programs

Note: older releases (1.0, 1.1 and 2.2) are unaffected.

Identifying affected programs

It should be possible to use the Totara system logs to identify programs affected by this issue via the following steps:

1. Find out the date and time your site was upgraded to 2.4.10 or 2.4.11.
2. Go to Site Admin > Reports > Logs
3. Generate logs for the dates where your site was affected, filtering by the 'update' action.
4. Search the 'Action' column for the string 'program update assignments'. Any records of this type while your site is on 2.4.10 or 2.4.11 will be affected. The information column should list the name of the affected program.
5. Clicking the 'program update assignments' link should open the assignments tab for the affected program so you can review.

 

Incident Report

What happened

We added a patch to convert the date pickers from 2-digit years to 4-digit years. This was required because there was ambiguity about when a two digit year was (i.e. 01/01/30 could be 1930 or 2030).

The patch we submitted modified the core English language pack to change some of the date related strings in the 'langconfig' and 'totara_core' files. This caused two problems:
1. Although the English language pack was updated, we did not update the equivalent strings in the downloadable language packs.
2. Because the same language pack is used for all sites on the same major version (e.g. 2.4.9 and 2.4.11) and our code change was only applied to versions after 2.4.10 there was actually no way to update the language packs to work in both cases.

The impact

1. The major impact of this is with program completion date assignments. Due to the way the assignment page was coded it was using two separate language strings to validate the dates - one from the core moodle language pack and one from totara's language pack. When using a downloaded language pack that was missing the change we applied the combination of these two strings causes validation to fail which results in all completion dates being cleared when the page is saved.

This bug should only impact sites using a downloaded language pack who saved a program assignments page after installing 2.4.10 or 2.4.11. Upgrading to 2.4.12 will allow assignments to be saved correctly again (including existing assignments). Unfortunately there is no way to retrieve the lost dates without a database backup of the site.

2. There are some other places that are affected but they are relatively minor - sites using a downloaded language pack where the date format strings are different from English will see the English date format instead of their languages date format. Examples of where this occurs include:
- Learning Plan template end date
- Learning Plan end date
- Learning plan item due dates
- Date based rules in audiences
- Dates displayed in some report sources

However all existing dates are still stored correctly, and users can change the dates using the datepicker and everything will still work fine.

If the user manually enters a date using their format they may end up with strange results. For example in US English if you manualkly entered '10/13/2000' (13th October 2000) it will be intepreted as 'the 10th day of the 13th month 2000', which is internally converted to 10th January 2001. This only happens if you manually enter the dates and ignore the hint text which would be in UK English format.

How this got through our testing process

The patch that introduced this bug went undetected through four stages of QA:

1. Automated unit testing
2. Manual code review by reading the code
3. Manual testing of the patch
4. Final integration testing of the whole release.

The primary reason we failed to pick up this issue was because none of our testing occurs in languages other than the core English language (in UK English the bug does not appear).

Totara mitigation actions

1. We will look at adding an automated test job in a non-English language. I don't believe it would have caught this particular issue but it may find other problems.

2. Ask our developers who speak other languages to routinuely use a non-English language pack when developing and testing.

3. Add a new section to our review policy requiring patches that modify 'langconfig' to be tested with other languages. Changes to the 'langconfig' strings are most likely to cause problems (as in this case) so warrent additional testing.

4. We will reiterate the importance of thorough testing prior to each release.

 

 

Simon Coggins
Re: Serious bug in 2.4.10 and 2.4.11
by Simon Coggins - Thursday, 17 October 2013, 6:34 PM
Group Totara

For sites affected by this issue, after upgrading to 2.4.12 please also take the following steps:

1. Click Site Admin > Languages > Language Packs

2. Click the "Update all installed language packs" button.

That should download the updated language pack which includes the new strings required by the fix.

Simon