Best practice forum (Archived)

Importing organisations via Totara sync

 
Umberto Zaccaro
Importing organisations via Totara sync
by Umberto Zaccaro - Friday, 5 April 2013, 3:50 PM
Group Partners

Hi,

I'm trying to import organisations by using Totara Sync.

After a lot of attempts I get this error:

"Impossible to upload all registers".

These are the rows in org.csv file:

deleted,idnumber,timemodified,fullname,shortname,description,frameworkidnumber,parentidnumber,typeidnumber

0,25,0,savignoni,sav,desc,A001,AM001,DM002
0,26,0,savignoni2,sav2,desc2,A001,AM001,DM002

Please, could you help me?

Thank you very much.

Kind Regards

Umberto

Umberto Zaccaro
Re: Importing organisations via Totara sync
by Umberto Zaccaro - Sunday, 7 April 2013, 3:28 PM
Group Partners

Hi,

in any case could you provide a example of correct file?

Thank you in advance.

Kind Regards

Umberto

Craig Eves
Re: Importing organisations via Totara sync
by Craig Eves (Totara Support) - Sunday, 7 April 2013, 3:53 PM
Group Totara

Hi Umbreto

The correct csv format is shown under Totara sync > Sources > Organisation

The fields that you choose under Totara sync > Elements > Manage elements are the fields displayed here.

An example of a file format for organisation is "idnumber","fullname","shortname","description","frameworkidnumber","parentidnumber","typeidnumber","timemodified"

The deleted field is not required,  this is chosen under Manage element > Remove elements. The frameworkidnumber  and typeidnumber needs to be defined - the sync does not create these.

Craig

Simon Coggins
Re: Importing organisations via Totara sync
by Simon Coggins - Sunday, 7 April 2013, 3:56 PM
Group Totara

Hi Umberto,

I've searched the code base and I can't find an example of the error:

"Impossible to upload all registers".

Was that the exact message you received? If not could you paste the exact error in?

Thanks,

Simon

Umberto Zaccaro
Re: Importing organisations via Totara sync
by Umberto Zaccaro - Monday, 8 April 2013, 3:59 AM
Group Partners

Hi All,

thank you for your support.

I made new tests to clarify this issue.

I tried to upload a file with standard format but I get this error:

"CSV file not valid, missing field "idnumber""

This error depends on first column in the file. If I change the "idnumber" with another first column as "fullname" I get this error:

CSV file not valid, missing field "fullname"

It seems does not able to view the first column in the file.

If I add, on the first column, "deleted" i get this error:

"could not import all records".

To complete this thread I tried these tests using command line :

php -f run_cron.php

If I try by using Run_Sync link I get this other error:

"cannot create directory: /var/www/vhosts/fideuram.epinoia.it/moodledata/sync/csv/store "

This message I think is wrong because the directory already exists and from command line works fine.

I attached file to clarify my issue.

I hope this helps to fix this issue.

Thank you very much indeed.

Kind Regards

Umberto

Joseph Muldoon
Re: Importing organisations via Totara sync
by Joseph Muldoon - Monday, 8 April 2013, 4:10 AM
 

Hi Umberto,

Have you checked the encoding of the file you are uploading? I have experienced a similar error message when the file was encoded to UTF-8, try encoding to ANSI.

To change the encoding of a file, use a program such as Notepad++ which is freely available.

Hope that helps.

Regards,

Joe

Umberto Zaccaro
Re: Importing organisations via Totara sync
by Umberto Zaccaro - Monday, 8 April 2013, 5:24 AM
Group Partners

Hi Joe,

great, this was exactly the error.

I'm wondering why, normally, the correct encoding would be UTF-8 but in this case with ANSI encoding CSV file works fine.

Thank you very much.

Kind Regards

Umberto

Alan Bradshaw
Re: Importing organisations via Totara sync
by Alan Bradshaw - Monday, 8 April 2013, 6:27 AM
 

I opened that file with a few different text editors. In all it looked OK but some pointed out it has a BOM which is not needed or recommended. One editor suggested the file may be broken. So I'm guessing that Totara or PHP doesn't handle the BOM correctly or it really is corrupted. 

The problem really is that some applications don't export UTF-8 very well. So if you're having this problem then try another encoding, as you have done.

Simon Coggins
Re: Importing organisations via Totara sync
by Simon Coggins - Monday, 8 April 2013, 7:02 PM
Group Totara

Hi Umberto,

Yes, you should remove the BOM mark from the file and it should work - totara_sync uses the standard PHP library functions for importing the CSV file and does support UTF-8 so most likely the BOM is the issue here.

Regarding your issue running totara sync from the command line - it may be that your data directory is owned by the web server user and the user running the command at the command line doesn't have write access but the web server does.

Normally I'd run it with a command like this:

sudo -u www-data php -f run_cron.php

which runs it as the "web server" user.

Simon