Manipulating wmlfiles by a script

[Shiki]

Between the Ageless Era and the Era of Myths, there are and will be a few differences in the wmlfiles.
While some things can be adjusted to be the same, some things will be every time different in the 150-200 unit files.
- the #textdomain line
- and that the unit graphics are in an different folder, that means that each line starting with
image="vampires/... will turn into image="units/vampires/... (for each faction)

While one approach may be to change that once, it has the problem that it may be hard to find the minor things which changed, like timings of animations, when you want to move the changes from the Era of Myths to Ageless, too.

I thought about how it is possible to make a script that makes these steps for you, and which you could let run over the Era of Myths after an newer version, instead to have to find and cherry-pick the changes. The points actually sound like what wmllint does for mainline things. I could also imagine some bash magic.
I don't know python, and not much about writing bash scripts either. Maybe you could help me with that?

[Ravana]

Generally I do that kind of stuff with notepad++ replace all in all opened. Then for some replacements I have used c++, back then it was the language I knew most.

Textdomain you may ignore here, ageless does not have translations.

I find it easiest to deal with external eras that have their own, unique enough prefix. Then I can just replace all of that.


Though, once has happened that it was too much. Archaic had changed so much at one point that I just removed ageless copy of it, brought new one from there, and did changes.


Animation timing I will only change if there is really vital need (https://github.com/ProditorMagnus/Agele ... bf9e5ff506).

[Elvish_Hunter]

I thought about how it is possible to make a script that makes these steps for you, and which you could let run over the Era of Myths after an newer version, instead to have to find and cherry-pick the changes.

That seems certainly possible.

While some things can be adjusted to be the same, some things will be every time different in the 150-200 unit files.
- the #textdomain line
- and that the unit graphics are in an different folder, that means that each line starting with
image="vampires/... will turn into image="units/vampires/... (for each faction)

Are these the only changes that you want to handle in the script?

I could also imagine some bash magic.
I don't know python, and not much about writing bash scripts either.

If you want to go for the Bash solution, sed might be your tool of choice. You can even use it with the find and xargs commands, and this will allow you to run the script only on files matching a certain regex.
For example, last December I cleaned the C++ source from trailing whitespaces. This was the command that I used:

Code: Select all

find <source directory> -name \*.\[ch\]pp -print0 | xargs -0 sed -i 's/[[:blank:]]*$//'

Otherwise, AWK is another suitable tool for that purpose, but I never needed to use it so far.

[Shiki]

The function of notepad++ to change all opened files was very useful.

After reading Hunters answer I went for the bash script, since it will be quicker.
It basically looks like this:

Code: Select all

find . -name \*.cfg -print0 | xargs -0 sed -i 's/#textdomain wesnoth-Era_of_Myths/#textdomain wesnoth-Ageless_Era/'
find . -name \*.cfg -print0 | xargs -0 sed -i 's/EOM_/AE_myh_/'
find units/vampires -name \*.cfg -print0 | xargs -0 sed -i 's/\"vampires\//\"units\/vampires\//'
... 

However, if there are multiple matches in a line, only the first gets changed. You can run the command multiple times, but exists there a fancier way?

Code: Select all

find <source directory> -name \*.\[ch\]pp -print0 | xargs -0 sed -i 's/[[:blank:]]*$//'

That's every time useful, will add it too.

Are these the only changes that you want to handle in the script?

There's one more. In the Ageless Era, there is every time a credits message in the unit description. It would be best to insert a macro before, but the problem with the way above is, that the attacks use the same description= key. However, the first appearance of that key in the file is always the unit description, while all further belong to attacks.
An alternative would be to have a macro for it in the Era of Myths with an empty string, and only give it in the Ageless Era the text, but I would prefer it in the script.

Textdomain you may ignore here, ageless does not have translations.

I think it would be very easy to add translations from the original eras, and if we have a script like mentioned above we don't need to take extra care for maintaining them either.

Animation timing I will only change if there is really vital need.

If we have an easy way to include the changes, I think there's no reason not to do so. In the git commit it will be visible what actually did change, and if I remember correctly git won't realize that we swapped the files. If you make other changes, I would happily add them too.

[Ravana]

There's one more. In the Ageless Era, there is every time a credits message in the unit description. It would be best to insert a macro before, but the problem with the way above is, that the attacks use the same description= key. However, the first appearance of that key in the file is always the unit description, while all further belong to attacks.
An alternative would be to have a macro for it in the Era of Myths with an empty string, and only give it in the Ageless Era the text, but I would prefer it in the script.

That is just what I did with c++

Code: Select all

#include<iostream>
#include<fstream>
#include<string>

using namespace std;

int main(){
    ifstream in("mrc_files.txt");
    ofstream out("temp.txt");
    string file_line,unit_line;
    bool type_open=false;
    bool other_open=false;
	while(getline(in, file_line))
	{
		ifstream unit_in(file_line);
		ofstream unit_out(file_line.replace( file_line.find("units"), 5, "out_u"));
		while(getline(unit_in, unit_line))
		{
			
			if(unit_line.find("[unit_type]") != std::string::npos){
				//cout<<unit_line<<"\n";
				type_open=true;
			} else {
				if((unit_line.find("[") != std::string::npos)&&unit_line.find("[/") == std::string::npos){
					other_open=true;
				}
			}
			if(unit_line.find("[/unit_type]") != std::string::npos){
				type_open=false;
			} else {
				if(unit_line.find("[/") != std::string::npos){
					other_open=false;
				}
			}
			if((unit_line.find("description=") != std::string::npos)&&(type_open==true)&&(other_open==false)&&(unit_line.find("This unit is from") == std::string::npos)&&(unit_line.find("_UNIT_NOTICE}") == std::string::npos)&&(unit_line.find("        ") == std::string::npos)){
				out<<unit_line<<"\n";
				string desc=unit_line.substr(unit_line.find('"'));
				unit_out<<"    description={AE_MRC_UNIT_NOTICE}+"<<desc<<"\n";
				
			} else {
				unit_out<<unit_line<<"\n";
			}
		}
	}
    return 0;
}

Basically, it should say that if [unit_type] is open, and nothing else is, then description is changed. But I wrote it some time ago and for only one-time use so it looks quite horrible.

[Elvish_Hunter]

However, if there are multiple matches in a line, only the first gets changed. You can run the command multiple times, but exists there a fancier way?

Sure there is It's the /g flag, which stands for "global replacement". This page explains quite well how it works:

Most UNIX utilities work on files, reading a line at a time. Sed, by default, is the same way. If you tell it to change a word, it will only change the first occurrence of the word on a line. You may want to make the change on every word on the line instead of the first.
[...]
If you want it to make changes for every word, add a "g" after the last delimiter...

So, for example, one of your lines becomes:

Code: Select all

find . -name \*.cfg -print0 | xargs -0 sed -i 's/EOM_/AE_myh_/g'

There's one more. In the Ageless Era, there is every time a credits message in the unit description. It would be best to insert a macro before, but the problem with the way above is, that the attacks use the same description= key. However, the first appearance of that key in the file is always the unit description, while all further belong to attacks.

That sounds like a complex case, because you'll need not only to check if you're inside a [unit_type] tag, but also that you're outside of an [attack] tag, then handle the description= key. That's where AWK might be better suited, as an AWK script can even set variables and execute conditional code.
For example, I quickly put together this small AWK script, which should print the description of a unit (it does on the Elvish Fighter), or at least the first line of the description. It isn't complete by any means (this is more or less my first time using it), but I suppose that it might be a decent starting point.

Code: Select all

#!/usr/bin/awk -f

BEGIN {
	in_unit_type = 0;
	in_attack = 0;
}

/\[unit_type\]/ {
	in_unit_type++;
}

/\[\/unit_type\]/ {
        in_unit_type--;
}

/\[attack\]/ {
        in_attack++;
}

/\[\/attack\]/ {
        in_attack--;
}

/[[:blank:]]description.?=/ {
	if (in_unit_type && (!in_attack)) {
		print;
	}
}

[Shiki]

About the C++ code, I didn't get it working.
All in all, it looks now like this.
It can be used with any era.
I think I will leave it like that.
Thank you for the help.

Code: Select all

#!/bin/bash

# change textdomain
find . -name \*.cfg -print0 | xargs -0 sed -i 's/#textdomain wesnoth-Era_of_Myths/#textdomain wesnoth-Ageless_Era/'

# change era präfix
find . -name \*.cfg -print0 | xargs -0 sed -i 's/EOM_/AE_myh_/g'

# the image files are in the ageless era in another folder
find units/celestials -name \*.cfg -print0 | xargs -0 sed -i 's/\"celestials\//\"units\/celestials\//g'
find units/devlings   -name \*.cfg -print0 | xargs -0 sed -i 's/\"devlings\//\"units\/devlings\//g'
find units/elementals -name \*.cfg -print0 | xargs -0 sed -i 's/\"elementals\//\"units\/elementals\//g'
find units/therians   -name \*.cfg -print0 | xargs -0 sed -i 's/\"therians\//\"units\/therians\//g'
find units/vampires   -name \*.cfg -print0 | xargs -0 sed -i 's/\"vampires\//\"units\/vampires\//g'
find units/wargs      -name \*.cfg -print0 | xargs -0 sed -i 's/\"wargs\//\"units\/wargs\//g'
find units/windsong   -name \*.cfg -print0 | xargs -0 sed -i 's/\"windsong\//\"units\/windsong\//g'

#remove trailing whitespaces
find . -name \*.cfg -print0 | xargs -0 sed -i 's/[[:blank:]]*$//'

# insert spaces if needed
find units -name \*.cfg -print0 | xargs -0 sed -i 's/description=_/description= _/'
find units -name \*.cfg -print0 | xargs -0 sed -i 's/description= _\"/description= _ \"/'

# Adds the unit notice
find units -name \*.cfg -print0 | xargs -0 insert_unit_notice -i inplace

Code: Select all

#!/usr/bin/awk -f

BEGIN {
   in_unit_type = 0;
   in_attack = 0;
}

/\[unit_type\]/ {
   in_unit_type++;
}

/\[\/unit_type\]/ {
        in_unit_type--;
}

/\[attack\]/ {
        in_attack++;
}

/\[\/attack\]/ {
        in_attack--;
}

/[[:blank:]]description.?=/ {
   if (in_unit_type && (!in_attack)) {
      sub("description=" , "description= {AE_myh_UNIT_NOTICE} + ");    
   }
}

{ print }