[Swlug] help with text manipulation

[Mark Einon]

Looks like the list doesn't like my attachments. Try this gist:

https://gist.github.com/einon/b954a386ac62473a6e46f38d3b555558

Mark


On Wed, 2018-06-20 at 09:02 +0100, Mark Einon wrote:
> Hi there,
> 
> This is simple to do in python (using the Pandas lib), I do it on a
> daily basis. A jupyter notebook in html and ipynb is attached to show
> how (you can use google's colabatory to edit and run the notebook - 
> https://colab.research.google.com/)
> 
> Mark
> 
> 
> On Wed, 2018-06-20 at 03:26 +0100, bascule via Swlug wrote:
> > this is a request for help, it's not actually linux related except
> > inasmuch as 
> > the solution may utilise a linux distro.
> > 
> > i have some json files i wish to edit, their being json files is
> > irrelevant to 
> > me, i'm not doing json stuff, these files come to me as they are and
> > i
> > need/
> > wish to futz with them.
> > 
> > they have objects in them such as:
> > 
> > {
> > "ConnectedObjects" : "",
> > "ExtraData" : "",
> > "FormID" : "004366",
> > "FormID_dec" : 184566630,
> > "PlotPath01FormID_dec" : 0,
> > "PlotPath01Plugin" : "",
> > "PlotPath02FormID_dec" : 0,
> > "PlotPath02Plugin" : "",
> > "PlotPath03FormID_dec" : 0,
> > "PlotPath03Plugin" : "",
> > "PlotPlanFormID" : "002151",
> > "PlotPlanFormID_dec" : 385884497,
> > "PlotPlanPlugin" : "SimSettlements_AddOnPack_Defences_Brae.esp",
> > "PlotStage" : 0,
> > "PlotVIPStoryFormID_dec" : 0,
> > "PlotVIPStoryPlugin" : "",
> > "RemoveHavok" : 0,
> > "Scale" : 1.0,
> > "idx" : 1471,
> > "isActor" : 0,
> > "isAssignable" : 1,
> > "isCreated" : 1,
> > "isFarmAnimal" : 0,
> > "isFood" : 0,
> > "isGenerator" : 0,
> > "isPowered" : 1,
> > "isTamedCreature" : 0,
> > "isTurret" : 0,
> > "isWater" : 0,
> > "name" : "Martial Plot - 2 x 2",
> > "plugin_name" : "SimSettlements.esm",
> > "posX" : -35624.8515625,
> > "posY" : 59097.19140625,
> > "posZ" : 3894.25146484375,
> > "rotX" : 0.0,
> > "rotY" : 0.0,
> > "rotZ" : -188.41806030273437
> > },
> > 
> > whitespace may have been lost a bit in the copy/paste, 
> > my files have thousands of such objects, i need to find all objects
> > where all 
> > of the the keys FormID,posX,posY,posZ,rotX,rotY,rotZ are the same as
> > another 
> > object's and delete all but one of the 'duplicated' objects
> > regardless
> > of 
> > whether any other key is the same.  i need to do this for objects in
> > the same 
> > file, and if possible for objects in two different files.
> > suggestions so far include learning python, and using jq, which man
> > page i 
> > have read and yes it seems to be a tool of sufficient power maybe,
> > but
> > i fear 
> > i lack the chops to work out how to do all the necessary streaming
> > and 
> > filtering, 
> > any other suggestions would be useful, 
> > 
> > bascule
> > 
> > 
> > 
> >