Bugs & Questions about Smart Copy

Job, no log file. The history on SmartCopy only shows what profile was updated. i don't query Geni for all the existing data in the Geni profile(s) or keep track of what the old value is on the Geni profile, but the Geni profile should have it in the revision history for most fields.

I could probably keep an optional log of what it submits to Geni.

I am getting the message "SmartCopy has been disabled from Chrome" and cannot enable? Temporary glitch?

Jeff.

You are right about the change history. There is no need to duplicate that. A list of updated/created profiles would be nice and that information that is not in the change history.

Would it be possible to add a possibility to use smartcopy to only add missing information to a profile and not make any updates to existing data?
Now you have to be very careful in checking to see if no changes where made that where incorrect, like the problems with expanding a place name to the wrong place. There may also be problems with the date fields. In the Netherlands we write dates as dd-mm-yyyy but sometimes also like dd/mm/yyyy and that last way of writing a date may be interpreted as an English data format.

SmartCopy Ap Link etc. re-instated after re-booting computer

Job, at the bottom of SmartCopy, there should be a link "Show History" which lists the last 50 updated/created profiles. It just doesn't include what was updated, but I'll look at adding that and we could increase it to more than 50.

By default, the only profile set to update is the focus profile and it only pre-checks when using smart matches and on content identified as new by MyHeritage, where it may say "Adds: middle name and birth date". The parents, siblings, children, and spouse default to an Add operation, so all data is new. You can manually set these to update existing Geni profiles, but I have no knowledge beforehand which Geni profile corresponds to what MyHeritage profile and I don't think trying to automatically match names is a good method.

So this brings up what Dan Cornett has been asking for, which is to create a side-by-side capability that compares the existing data, which for relatives would display as you select the Geni profile it matches. It's a good idea, but it will be a bit off work to implement and would have a performance impact as I'd have to query all the Geni data from all the relatives. But I could start on the focus profile and try to work through it.

On the dates, most of the sites we're pulling from are not free form. The way the date is presented is defined in the software and should be consistent to some degree for each site. I've set some templates to look for, but I didn't write the date parse myself - I use a third party script, http://momentjs.com/

Jeff,

Thanks Jeff.
If Smart Copy adds a lot of profiles and/or a lot of data to existing profiles the least that is needed is a way to make sure exactly what was added/changed. So users that are interested can check and correct any mistakes or conflicts.

A side by side compare would be very nice. It would be even better if you had to confirm on a per profile basis any add/change Smart Copy would like to make. It will make copying slower, but I think that will be for the best.
I'm rather afraid that any bulk processing with Smart Copy will introduce at lot of problems if you would have to rely on its user to check and correct after Smart Copy had finished its actions.

I see a lot of merges on Geni where users did not correct tree conflicts and even more where they did not solve data conflicts. I hope that Geni will make such conflicts visible during a merge and not rely on users to check afterwards.
I'm afraid the same will happen with Smart Copy if it does not ask for user input before adding a profile or making a profile update.

I agree that matching names would be very tricky, but you could may be warn a user that Smart Copy would add a second pair of parents or a second marriage on the same day or a spouse with a same date as an already present spouse ore a child that is born within 9 months after another already present child (or ...).

It may be slower to have to read the old data from Geni, but correcting mistakes takes much more time and effort (especially when it has to be done by others, because the Smart Copy user did not bother)

The date problem might be tricky, especially where parts of the date are unknown. I did not test it with Smart Copy, but I notices users use different ways (depending on which program(s) they tend to use and what is allowed)
I've seen things like 0 or 00 for an unknown value and ? or ?? for a value that is missing or could not be read, but many more variations (it gets even more complicated when using BC dates or dates with a different starting point.)

There is another problem with Dates I noticed on Geni. I find when merging profiles that there are a lot of profiles for the same person that do not agree with part is the day and which part is the month (they are reversed in the merge profiles). This is probably due to human error (English date format differs from most other European date formats on month and day).
Probably some of the input dates for Smart Copy may also have month and day reversed. If you do not check the old value in Geni, but just overwrite it, that may go unnoticed by the user.

I'm a bit short on time at the moment, but will try to test a bit more with incomplete place names. How should that work at the moment? (Would the result be depended on Geni languages for the profile, Geni language setting for the user, operating settings for the user, would there be a dependency on any setting on the source platform?).

For those who want to test with MyHeritage data without having to be a paying member on MH:
I'm a guest user on several MH sites and that allows me to see MH tree matches in Geni. For those who want to test Smart Copy with real MH data without having to pay for MH, the way would be to get a paying MH user to invite them to their site.

Sorry forgot to mention:
If Smart Copy only copies new content, so I would not overwrite existing dates, there still could be a problem with birth/baptise and death/buried dates. A lot of users will put a baptise date in a birth field an/or a burial date in a death date. If the Geni profile only has the other fields filled, you would fill the empty dates with Smart Copy. A reversal of day and month or an incorrect translation of a missing date part may be hard to spot afterwards.

Job, just so I understand your perspective. Are you using SC as a basic member to update profiles?

Jeff,

Yes, that is correct.

[[6000000034219976799]

Smart Copy shows spouse, no children.
Entered children. Smart Copy shows parents in the children's profiles, no siblings.

When I entered data in a profile and save if parents exist, they are shown as a match in Smart Copy. Once I enter more profiles linked to this profile, only the parents and/or spouse is shown in Smart Copy, no siblings or children.

When I enter siblings and children, only parents show as a match in Smart Copy, no siblings or children.

This seems to be the case for newly created profiles as I went back to a previously entered profiles (last week's entries) and the problem does not seem to exist.

It seems the problem has been fixed. Thank you.

Job, I'll try to answer these in sections...

--> "If Smart Copy adds a lot of profiles and/or a lot of data to existing profiles the least that is needed is a way to make sure exactly what was added/changed. So users that are interested can check and correct any mistakes or conflicts."

I just released version 1.53 which allows you to expand each profile in "Show History" section of the tool to see what you submitted to Geni, minus the About wiki text (too long). This data is not retroactive, so you'll only see the option for new profiles added / updated. I also increased the history from 50 profiles to 100 profiles.

I can't, however, do much for what other users see. Apart from detailing the changes in the About text, which I won't do, the only way to see the revisions will be Geni's revision system. The Geni revisions do cover most of the fields SmartCopy updates, so I don't see this as an issue. Exceptions may be the marriage date / location if you update a spouse record.

There shouldn't be any conflicts with SmartCopy submissions unless the user adds a new profile and then merges it with an existing profile. The merging process and conflict resolution is outside the scope of SmartCopy and should be handled like any other merge.

--> "A side by side compare would be very nice. It would be even better if you had to confirm on a per profile basis any add/change Smart Copy would like to make. It will make copying slower, but I think that will be for the best.
I'm rather afraid that any bulk processing with Smart Copy will introduce at lot of problems if you would have to rely on its user to check and correct after Smart Copy had finished its actions."

--> "I see a lot of merges on Geni where users did not correct tree conflicts and even more where they did not solve data conflicts. I hope that Geni will make such conflicts visible during a merge and not rely on users to check afterwards.
I'm afraid the same will happen with Smart Copy if it does not ask for user input before adding a profile or making a profile update."

The purpose of SmartCopy is bulk processing, so I'm not favorable to confirmation for each profile. For the most part, it's function is adding profiles to Geni. Updating immediate family profiles is a secondary function that was worth including, but should have limited overall use. When using it, users should be checking beforehand what data needs to be added, comparing the level of detail in the Geni profiles to those under review and updating only those fields that bring improved data. Ultimately, it may not save them any time from doing it manually. When I've used it, it's often something like just uploading pictures or updating fields that I know the Geni profiles don't have data for, such as death date or burial location.

As we say in the Project "Usage & Access" and in the message we send when Curators authorize someone, the user needs to be responsible with the tool and we reserve the ability to revoke that use. I guess one loophole we have here is that I haven't required authorization for Basic members or the ability to revoke their use. It only applies to Pro members because the concern was irresponsible duplication, so the authorization and revocation applies to adding profiles to Geni via SmartCopy. At the time, Basic members could only update the focus profile, so there was little potential harm, but now I've allowed it to update any of the family members as well. So I guess the question I need to pose to the Curator group is if we should require authorization for Basic members as well for potential irresponsible updates. Or perhaps, not authorization but allow them the ability to revoke family member updates for Basic users.

--> "I agree that matching names would be very tricky, but you could may be warn a user that Smart Copy would add a second pair of parents or a second marriage on the same day or a spouse with a same date as an already present spouse ore a child that is born within 9 months after another already present child (or ...)."

--> "It may be slower to have to read the old data from Geni, but correcting mistakes takes much more time and effort (especially when it has to be done by others, because the Smart Copy user did not bother)"

SmartCopy defaults to update on parents when existing parents exist. The user can change this if they're adding adoptive parents. Often SmartCopy is being used with MyHeritage matches, so I use their scoring to determine if there is a new spouse, children, or siblings. Otherwise, I don't precheck anything. That is up to the user to look at Geni in another tab and then choose who to add based on what is already there. All that comparative logic would be great but it's a good bit of work and I'm not sure if it's worth the time it would take. My wife is already on me about how much volunteer time I donate to Geni and making free tools when I could be working and allow her to retire. :)

Ultimately, SmartCopy should be used by responsible users and while I'm fine with making things easier, I'm not going to be able to spend the time to program away someone that's just not talking the time to review what they're submitting. SmartCopy was designed to save time by automating what a user would have to do manually. If the user is not being responsible with it, then it only takes me a moment to send them a finger waving message or revoke their use of the tool for a period of time.

--> "The date problem might be tricky, especially where parts of the date are unknown. I did not test it with Smart Copy, but I notices users use different ways (depending on which program(s) they tend to use and what is allowed)
I've seen things like 0 or 00 for an unknown value and ? or ?? for a value that is missing or could not be read, but many more variations (it gets even more complicated when using BC dates or dates with a different starting point.) "

--> "There is another problem with Dates I noticed on Geni. I find when merging profiles that there are a lot of profiles for the same person that do not agree with part is the day and which part is the month (they are reversed in the merge profiles). This is probably due to human error (English date format differs from most other European date formats on month and day).
Probably some of the input dates for Smart Copy may also have month and day reversed. If you do not check the old value in Geni, but just overwrite it, that may go unnoticed by the user. "

--> "If Smart Copy only copies new content, so I would not overwrite existing dates, there still could be a problem with birth/baptise and death/buried dates. A lot of users will put a baptise date in a birth field an/or a burial date in a death date. If the Geni profile only has the other fields filled, you would fill the empty dates with Smart Copy. A reversal of day and month or an incorrect translation of a missing date part may be hard to spot afterwards."

Most dates that I've seen on sites that SmartCopy supports don't use the confusing formats that you're referencing. They say "July 2, 1945" or something that clearly distinguishes the day and month. The ones that do - maybe rootsweb?, they were consistent in their use and have resolved accurately. I don't expect SmartCopy to be used for anything in the BC range. It's target use is Present to maybe 1500. Anything before that date should already be on Geni. I try to filter out things like ?? and if I can't figure out the date, I just don't include it. If you find one that is not formatting properly, let me know and I'll see what I can do.

--> "I'm a bit short on time at the moment, but will try to test a bit more with incomplete place names. How should that work at the moment? (Would the result be depended on Geni languages for the profile, Geni language setting for the user, operating settings for the user, would there be a dependency on any setting on the source platform?)."

SmartCopy sends all location data to Google, which attempts to Geolocate the place and break it down to it's field elements. Google is not perfect at this, so you have the option to turn off location detection and just put the entire string in the Place field. All results should translate to English regardless of Geni, browser, or OS settings.

Jeff,

Thanks. I appreciate you taking the time to answer in such detail.
I will test some more and will let you know when I find any problems.

I'm also a big fan of history link. It help me a lot to find new data.
I think Smart Copy will be very useful even for me as non paying Geni and MH user. I hope to repay you some by testing Smart Copy.

Actually, I just noticed that SmartCopy doesn't offer basic users the ability to update the family members, just the focus profile. So this is something I could look at adding. I'll run it by the other curators and see what they think.

Curators FYI - SmartCopy will update locked fields (curators only), so be cautious when updating an MP.

v1.55 - Given a sibling or child is parsed as Gender 'Unknown'; you know there is a match, and you set the Gender correctly, then select the proper Geni name so that the detail item says 'Update <geni-name>'. The icon does *not* change from the plus to the circle.

(It seems to behave with the updating correctly, but does not change the icon in that specific instance where the gender is set/changed explicitly. Doesn't really matter if the original Gender was 'Unknown' or not ... just that it was explicitly changed from the original parsed value.).

Thanks Dan - I've pushed an update to fix it.

Private User or Dan Cornett,

How do you update the extension using Google Chrome?

Kevin Lawrence Hanit It should update automatically at some point, but you can do it manually by right clicking on SmartCopy and selecting "Manage Extensions". Check the box at the top that says "Developer mode" and and click the button "Update extensions now" that appears.

Private User,

Thanks for the information. I have now updated to the latest version.

.... the update 1.55.1 does fix that visual add/update bug. Thanks, Jeff.

Private User,

Cannot get this one to load in SmartCopy:

http://www.myheritage.com/research/collection-1/myheritage-family-t...

Kevin

Private User,

I take that back as it was able to load.

Instead, I would like you to have a look at how the name is parsed. It did not parse correctly, so I went ahead and fixed it before letting SmartCopy do its work.

Kevin: That looks like an odd-ball input data... they put "1 St SGT" as part of the name.

SmartCopy has some prefixes (titles) it tries stripping off (and putting in the Display Name, if any found), but I'm not sure this prefix is common enough to add to the list.

I suppose one could strip off any numbers in front of the name, but in this case that would still leave "St Sgt ......"

Dan Cornett,

Here is a suggestion.

Why don't you strip out the number and the related ' St.' or 'St.' before parsing it and replacing it with '1st'?

Kevin

Hi. I have not used SmartCopy since Ancestry changed their Tree View significantly. Not sure if that's why but I get only a general "Cannot Parse" message now... Thanks

As I mentioned it's a general result but here is one profile http://person.ancestry.com/tree/71349460/person/30234995204/story