[Swlug] help with text manipulation

[David Hodgkinson]

That’s a few lines of perl...

> On 20 Jun 2018, at 03:26, bascule via Swlug <swlug at mailman.lug.org.uk> wrote:
> 
> this is a request for help, it's not actually linux related except inasmuch as 
> the solution may utilise a linux distro.
> 
> i have some json files i wish to edit, their being json files is irrelevant to 
> me, i'm not doing json stuff, these files come to me as they are and i need/
> wish to futz with them.
> 
> they have objects in them such as:
> 
> {
> "ConnectedObjects" : "",
> "ExtraData" : "",
> "FormID" : "004366",
> "FormID_dec" : 184566630,
> "PlotPath01FormID_dec" : 0,
> "PlotPath01Plugin" : "",
> "PlotPath02FormID_dec" : 0,
> "PlotPath02Plugin" : "",
> "PlotPath03FormID_dec" : 0,
> "PlotPath03Plugin" : "",
> "PlotPlanFormID" : "002151",
> "PlotPlanFormID_dec" : 385884497,
> "PlotPlanPlugin" : "SimSettlements_AddOnPack_Defences_Brae.esp",
> "PlotStage" : 0,
> "PlotVIPStoryFormID_dec" : 0,
> "PlotVIPStoryPlugin" : "",
> "RemoveHavok" : 0,
> "Scale" : 1.0,
> "idx" : 1471,
> "isActor" : 0,
> "isAssignable" : 1,
> "isCreated" : 1,
> "isFarmAnimal" : 0,
> "isFood" : 0,
> "isGenerator" : 0,
> "isPowered" : 1,
> "isTamedCreature" : 0,
> "isTurret" : 0,
> "isWater" : 0,
> "name" : "Martial Plot - 2 x 2",
> "plugin_name" : "SimSettlements.esm",
> "posX" : -35624.8515625,
> "posY" : 59097.19140625,
> "posZ" : 3894.25146484375,
> "rotX" : 0.0,
> "rotY" : 0.0,
> "rotZ" : -188.41806030273437
> },
> 
> whitespace may have been lost a bit in the copy/paste, 
> my files have thousands of such objects, i need to find all objects where all 
> of the the keys FormID,posX,posY,posZ,rotX,rotY,rotZ are the same as another 
> object's and delete all but one of the 'duplicated' objects regardless of 
> whether any other key is the same.  i need to do this for objects in the same 
> file, and if possible for objects in two different files.
> suggestions so far include learning python, and using jq, which man page i 
> have read and yes it seems to be a tool of sufficient power maybe, but i fear 
> i lack the chops to work out how to do all the necessary streaming and 
> filtering, 
> any other suggestions would be useful, 
> 
> bascule
> 
> 
> 
> 
> -- 
> Swlug mailing list
> Swlug at mailman.lug.org.uk
> https://mailman.lug.org.uk/mailman/listinfo/swlug