Write mp server bots in lua

Discussion among members of the development team.

Moderator: Forum Moderators

Post Reply
User avatar
iceiceice
Posts: 1056
Joined: August 23rd, 2013, 2:10 am

Write mp server bots in lua

Post by iceiceice »

Hi,

I have had an idea for some time that it would be useful to extend the lua scripting built into the game, to also allow scripting of things like mp lobby interaction, including sending and receiving chats, starting a game, joining a game, exiting a game. Basically, while the existing lua kernel is meant to script changes to the game state of a running game, I would also like to be able to script some of the user interface actions. There are several things that this would achieve:
  1. Make chat bots easily.
  2. Make "smart" bots that can configure a custom scenario with an AI. This might be really helpful if someone has made a custom AI and wants to get people to play it. Right now, your best option is to work in lua, and then distribute your AI as an add-on and try to get people to download it. I would like to be able to just have a bot that idles in the server, and when someone whispers to it "icebot: let's play a game" for example, it will host a game configuring the custom ai correctly for them, and act as the host, serving that ai's moves to them. This will work even if it's a C++ AI since the other client won't need to have the actual AI code.
  3. Make it easier to have little tournaments. For instance I'd like for another bot to be able to say "icebot: let's play 50 games" and have icebot automatically save the results. That way all that would have to happen to have a tournament is, everyone would have to make their bot join the mp server at some designated time, and the tournament organizer can run the tournament and treat them all as black boxes. It also avoids some problems with compatibility -- the tournament manager isn't responsible to run the bots or even gather their codes, he just has to collect the names of the bots in the tournament, and run a script that asks them all to play eachother in whatever order. If an ai has a bug and crashes or something, it won't crash the tournament program, it will just cause that bot to disconnect and time out and the situation presumably can be handled more simply by the non-crashed bots.
  4. Make it possible to unit test some of the mp networking code, by checking if bots can successfully play test scenarios together.
  5. It might be useful for improving third-party support of mp ladders. I remember in some other thread fendrin was talking about having an API of some kind to add support for third party ladders. I don't remember what he had in mind but I imagine it was to do things like, report wins, validate identities, etc. A ladder bot might be able to do these things, and also set up official matches. I guess beyond this there could be like "tournament" bots for tournaments with humans, so for instance if there is a scheduled tournament like TGT, there could be a bot that idles in the server and tells people when the next match is etc. (this is a bit extravagant though...).
Assuming I've sold anyone on the merits of this, let me explain roughly how I thought this would work:
  1. In the wesnoth lua API, separately from but similarly to the wesnoth proxy table, I would like to add a new proxy table named (tentatively) multiplayer, which would provide methods like multiplayer.chat, multiplayer.create_game, multiplayer.join_game, multiplayer.lobby_game_list, multiplayer.lobby_user_list, etc. My idea is that lobby.create_game should basically just take a lua table corresponding to the desired game config as its argument. That is, the work normally done by the mp create and mp configure dialogs would be done directly in lua by the bot, which would assemble a lua table corresponding to a wml config in the usual way, which would correspond to the full "game_config" for the scenario. It might be that some parts would be easier if there were lua helper functions to, say, find a modification from the config_cache and plug it into a partially completed game config, but anyways it seems tractable either way. The other functions like this should be relatively straightforward.
  2. There should be some lua interface to the mp connect dialog I suppose. The most rudimetary thing would be to be able to get the current game config for the scenario, so you can see who is assigned to what slot. Then you would also want to be able to ask whether we are ready to start, and have a function to either start or cancel.
  3. The whole thing would be event-driven. A bot would hypothetically consist of perhaps three function definitions -- an initializer where you load any config data files / a helper library, a destroyer where you perhaps save a log or some bot state or something just before you exit, and a "main loop" function which reacts to events. Events will be formatted as lua tables, and these tables will always just be basically the lua table translation of some wesnoth config which was sent over the network, like [chat] speaker=... message=... [/chat]. The bot might react to a message by first making a query maybe like multiplayer.current to figure out e.g. are we in the lobby? are we in mp_connect? are we in a game right now?, or some lua state variables that it is maintaining.
  4. The bot definition would ultimately be some lua file. You would launch it by passing command line arguments -- e.g.

    Code: Select all

    ./wesnoth --mp-bot icebot.lua --server baldras.wesnoth.org --username icebot --password secret 
    I guess I imagine that you would want to run headlessly but maybe for debugging purposes it could be useful to have the gui, so you can also type and do stuff while the bot is running by bringing up the window, maybe send some chats and then exit a game, supposing it has run amok or something.
This is all still quite sketchy, but it seems well-formed enough that I could start soliciting feedback. In particular:
  1. Does this design sound reasonable and non-ugly, at least from the lua perspective? I have made various commits that add things to the current lua API of the game, but I can't say that I've worked extensively in lua or have a good appreciation for it. My commits to the lua side of things are pretty small, and didn't require a deep understanding of the lua implementation. Is there some better lua idiom to achieve what I'm describing that should be used instead?
  2. Technical challenge -- the current Lua Kernel is managed by the "game_events::manager" object, which basically ties it to the WML events queue. Every time a game ends the lua kernel is destroyed, and a new one is created when a new game starts. As far as I know there is no lua kernel in between games. Not all of the game resources work this way -- some of them are created during the first game, then not destroyed and instead preserved for future games. For instance I think the display object works this way if I remember correctly (at least it seemed this way when I was debugging some stuff involving the wesnoth help viewer when accessed from the main screen... it was a while ago, I would have to look at it again). As I understand, lua kernels are meant to be fast and have a small footprint. To my mind the simplest thing might be to have two lua kernels, a long-lived one which persists across games and runs the bot (if there is one -- maybe this one just never gets constructed unless there's a bot), and the other which we currently have and would be unchanged. This seems cleaner since it sandboxes the bot to some extent, which seems very desirable. I'm not sure if there are unacceptable performance consequences of this though, or if there are nasty singleton objects in the lua implementation that would prevent this from working properly.
  3. I think it's probably useful if the bot can have readonly access to the wesnoth proxy table somehow so it can look at the game state, but it might be better to skip that if it allows the possibility of game state corruption which could be quite annoying. Maybe basically have two independent APIs essentially, the "bot" one with the multiplayer proxy table, which don't have access to the wesnoth proxy table, and keep the current lua api used by scenarios exactly as it is, without exposing the multiplayer proxy table.
  4. Anything else???
Anyways, still clearly very preliminary... any thoughts or crits will be most appreciated :)

I probably won't actually break ground on this for several weeks, because I have a backlog of other things to finish, but you never know. Anyways don't feel you need to rush to convince me not to do something hideous to the codebase. I would very much like to think this through carefully before doing anything...

~iceiceice~
User avatar
iceiceice
Posts: 1056
Joined: August 23rd, 2013, 2:10 am

Re: Write mp server bots in lua

Post by iceiceice »

This was PM'd to me, because of the forum restriction on developer discussions. Maybe should have posted in a different forum... :hmm:
Ravana wrote:Subject: Write mp server bots in lua
iceiceice wrote:Hi,

I have had an idea for some time that it would be useful to extend the lua scripting built into the game, to also allow scripting of things like mp lobby interaction, including sending and receiving chats, starting a game, joining a game, exiting a game. Basically, while the existing lua kernel is meant to script changes to the game state of a running game, I would also like to be able to script some of the user interface actions. There are several things that this would achieve:
...
Anyways, still clearly very preliminary... any thoughts or crits will be most appreciated :)

I probably won't actually break ground on this for several weeks, because I have a backlog of other things to finish, but you never know. Anyways don't feel you need to rush to convince me not to do something hideous to the codebase. I would very much like to think this through carefully before doing anything...

~iceiceice~
Since I do not have post access in that topic I send my reply here, if you want you can include that in topic.
To begin with, I fully support this idea. Before I thought that only way for lobby bot was locally compiling wesnoth from different files, oos wouldn't matter there as I was not that ambitious to expect game managing bots.

In lobby lists bots should look different from both registered users and guests.

Lua has possibility for sockets, so bot could be tied to other functions that are not in game, for example being irc bot too and being controlled from there.

Most I have used lua so far is few lines in scenarios for which wml lacked capability, so I cannot say how good this technical solution could be. Running bot from command line is good way.

And finally: ways to abuse this. Most visible way would be spamming. This could be fixed by finishing implementing /room, /join and /part commands and not allowing lobby messages without some checks, for example about lenght*frequency. Similar checks for private messages. Then bot could use its own room(channel), sending messages only to those who want them.
Edit: Some thoughts:

Yeah so in principle, you could just write a chat bot in any language, because the protocol to talk to the server is pretty simple and is documented on the wiki somewhere MultiplayerServerWML

If you want it to control games, understand what's going on and host an AI, then building it into the wesnoth executable this way seems better. I think some very basic possibility of this would be required for unit tests anyways. If we intend to do that, then doing it via lua scripting will kill all of these birds with one stone. It's possible that other parts of the game interface could be scripted as well, so eventually it might be extended, I haven't thought about it much.

I like the idea that the bot can be on irc and also on the mp server at the same time, that seems nifty.

About abuse: I guess at some point we drafted some policy rules about bots here MultiplayerServerBots, but it looks like it wasn't finalized, and as far as I know there are no mp server bots anyways right now.
User avatar
iceiceice
Posts: 1056
Joined: August 23rd, 2013, 2:10 am

Re: Write mp server bots in lua

Post by iceiceice »

There is an update on this, a preliminary version of this has been merged and is being run in our unit test suite. Two bots that make a test game and play it look like this right now:

https://github.com/wesnoth/wesnoth/blob ... 1/host.lua
https://github.com/wesnoth/wesnoth/blob ... 1/join.lua

I'm not going to make documentation right now because its not set in stone but I'll give an overview.

The key to the whole system is the lines like these:
https://github.com/wesnoth/wesnoth/blob ... st.lua#L27

Code: Select all

events, context, info = coroutine.yield()
When lua is executing the script, when it encounters one of these lines, the script will pause until the next time slice. The next time the script runs, the script will resume at this line, and the variables events, context, info will be updated with info about where the game currently is and actions that can be executed.

events is a list of notifications of events that happened, an array of wml tables from the engine. For instance this line checks for a chat message "ready" that was sent since the last time slice: https://github.com/wesnoth/wesnoth/blob ... st.lua#L72

info is a table of functions, representing queries about the current state. For instance, in this line: https://github.com/wesnoth/wesnoth/blob ... in.lua#L17
info.gamelist() requests a table corresponding to the gamelist for the current lobby. info.name is a string which names the current context.

context is a table of functions, representing actions that could be taken, which might change the state. For instance this line:
https://github.com/wesnoth/wesnoth/blob ... st.lua#L32

When an action is executed, the game might move to a different context, and old actions may become invalid or meaningless. When a script calls for an action, it doesn't actually get executed until the next call to coroutine.yield(). If many actions were requested before a yield, they are executed in order, and if the context changes as a result of one, the rest of them are discarded. After coroutine.yield() is called, all of the functions in the old context, info become invalid, the script should discard them and use only the new ones. It won't crash the game if you call an old function, but it will be a lua error.

This system is quite simple and flexible, and it's not too difficult to extend it to add new bindings to different parts of the engine.

Right now there are very minimal bindings to be able to create a game, skip through the configure screen, and start a game from the ready dialog, as well as browse the lobby and exchange chat messages. It is also possible to request the lua "package" package, which allows your script to load arbitrary packages from the operating system. I think this should allow you to bind the script to an external program, although I haven't tested this.

Edit:

The simplest way to make a script that does something right now (this is also what I did, since I didn't memorize what binding functions are in which contexts...) is like this:
  1. Start by opening a function which is the plugin:

    Code: Select all

    local function plugin()
    end
    
    return plugin
    
    End the script by returning the function.
  2. Declare whatever local variables, helper functions, and helper libs

    Code: Select all

    local function plugin()
    
      local function log(text)
        std_print("join: " .. text)
      end
    
      local events, context, info
    
      local counter = 0
    
      local helper = wesnoth.require("lua/helper.lua")
    
      local function idle_text(text)
        counter = counter + 1
        if counter >= 100 then
          counter = 0
          log("idling " .. text)
        end
      end
    
      log("hello world")
    end
    
    return plugin
    
  3. Add a loop which runs until you get to a good context. (The first context may not be title screen, it might be an "init" context...)

    Code: Select all

     repeat
        events, context, info = coroutine.yield()
        idle_text("in " .. info.name .. " waiting for titlescreen or lobby")
      until info.name == "titlescreen" or info.name == "Multiplayer Lobby"
    
  4. Add a call to the lua console

    Code: Select all

      wesnoth.show_lua_console()
    
    Now, run the script and inspect the variables context and info to see what you can do.
  5. Add some logic to use the available functions to navigate to a new context perhaps. Move the call to wesnoth.show_lua_console to the end.

    Code: Select all

    
      while info.name == "titlescreen" do
        context.play_multiplayer({})
        log("playing multiplayer...")
        events, context, info = coroutine.yield()
      end
    
      repeat
        events, context, info = coroutine.yield()
        idle_text("in " .. info.name .. " waiting for lobby")
      until info.name == "Multiplayer Lobby"
    
      wesnoth.show_lua_console()
    
    Now test it and see if the results were what you expected.
  6. Repeat.

    Code: Select all

      context.chat({message = "hosting"})
      log("creating a game")
      context.create({})
    
      repeat
        events, context, info = coroutine.yield()
        idle_text("in " .. info.name .. " waiting for create")
      until info.name == "Multiplayer Create"
    
      wesnoth.show_lua_console()
    
    etc...
Post Reply