[Swlug] help with text manipulation

[bascule]

On Friday, 22 June 2018 16:31:53 BST bascule via Swlug wrote:
>  then removed all but two entries in the json file to see if i could get the
> script to run without error, unfortunately unless i removed the
> (subset=....) part of the script i would always get:
> KeyError: 'FormID'

i created a new test.json by adding a duplicate entry and removing the 'items' 
part, i had to use an online json validator to make it right, this throws no 
error, but doesn't drop the duplicate entry
test2.json:

[
{
            "ConnectedObjects" : "",
            "ExtraData" : "",
            "FormID" : "239530",
            "FormID_dec" : 2331952,
            "RemoveHavok" : 0,
            "Scale" : 1.0,
            "idx" : 0,
            "isActor" : 0,
            "isAssignable" : 0,
            "isCreated" : 0,
            "isFarmAnimal" : 0,
            "isFood" : 0,
            "isGenerator" : 1,
            "isPowered" : 0,
            "isTamedCreature" : 0,
            "isTurret" : 0,
            "isWater" : 0,
            "name" : "Generator - Windmill",
            "plugin_name" : "Fallout4.esm",
            "posX" : 48868.17578125,
            "posY" : 39018.88671875,
            "posZ" : 371.94830322265625,
            "rotX" : 0.0,
            "rotY" : 0.0,
            "rotZ" : 94.794990539550781
},
{
            "ConnectedObjects" : "",
            "ExtraData" : "",
            "FormID" : "239530",
            "FormID_dec" : 2331952,
            "RemoveHavok" : 0,
            "Scale" : 1.0,
            "idx" : 0,
            "isActor" : 0,
            "isAssignable" : 0,
            "isCreated" : 0,
            "isFarmAnimal" : 0,
            "isFood" : 0,
            "isGenerator" : 1,
            "isPowered" : 0,
            "isTamedCreature" : 0,
            "isTurret" : 0,
            "isWater" : 0,
            "name" : "Generator - Windmill",
            "plugin_name" : "Fallout4.esm",
            "posX" : 48868.17578125,
            "posY" : 39018.88671875,
            "posZ" : 371.94830322265625,
            "rotX" : 0.0,
            "rotY" : 0.0,
            "rotZ" : 94.794990539550781
},
{
            "ConnectedObjects" : "",
            "ExtraData" : "",
            "FormID" : "030386",
            "FormID_dec" : 197510,
            "RemoveHavok" : 0,
            "Scale" : 1.0,
            "idx" : 1,
            "isActor" : 0,
            "isAssignable" : 0,
            "isCreated" : 0,
            "isFarmAnimal" : 0,
            "isFood" : 0,
            "isGenerator" : 1,
            "isPowered" : 0,
            "isTamedCreature" : 0,
            "isTurret" : 0,
            "isWater" : 0,
            "name" : "Generator - Small",
            "plugin_name" : "Fallout4.esm",
            "posX" : 48012.734375,
            "posY" : 40694.76953125,
            "posZ" : 630.52374267578125,
            "rotX" : 0.0,
            "rotY" : 0.0,
            "rotZ" : 204.53659057617187
}
]

output.json:
[{"ConnectedObjects":"","ExtraData":"","FormID":239530,"FormID_dec":
2331952,"RemoveHavok":0,"Scale":1,"idx":0,"isActor":0,"isAssignable":
0,"isCreated":0,"isFarmAnimal":0,"isFood":0,"isGenerator":1,"isPowered":
0,"isTamedCreature":0,"isTurret":0,"isWater":0,"name":"Generator - 
Windmill","plugin_name":"Fallout4.esm","posX":48868.17578125,"posY":
39018.88671875,"posZ":371.9483032227,"rotX":0,"rotY":0,"rotZ":94.7949905396},
{"ConnectedObjects":"","ExtraData":"","FormID":239530,"FormID_dec":
2331952,"RemoveHavok":0,"Scale":1,"idx":0,"isActor":0,"isAssignable":
0,"isCreated":0,"isFarmAnimal":0,"isFood":0,"isGenerator":1,"isPowered":
0,"isTamedCreature":0,"isTurret":0,"isWater":0,"name":"Generator - 
Windmill","plugin_name":"Fallout4.esm","posX":48868.17578125,"posY":
39018.88671875,"posZ":371.9483032227,"rotX":0,"rotY":0,"rotZ":94.7949905396},
{"ConnectedObjects":"","ExtraData":"","FormID":30386,"FormID_dec":
197510,"RemoveHavok":0,"Scale":1,"idx":1,"isActor":0,"isAssignable":
0,"isCreated":0,"isFarmAnimal":0,"isFood":0,"isGenerator":1,"isPowered":
0,"isTamedCreature":0,"isTurret":0,"isWater":0,"name":"Generator - 
Small","plugin_name":"Fallout4.esm","posX":48012.734375,"posY":
40694.76953125,"posZ":630.5237426758,"rotX":0,"rotY":0,"rotZ":204.5365905762}]

as you can see this is just the original test2.json, minus formatting:)
bascule