Apr 13th 2021, 20:33:02
Development on the NPCs is still ongoing. The following items are already live on Express:
1) Better destocking
2) Better NPC logging and reporting
3) Better public market purchasing
Going back to the other post, these items are remaining:
4) Better public market selling
5) Misc issue clean up
6) Add retals for bots
This post is about development that's still in progress, so the details might change. Much of my focus lately has been on improving public market selling, which is a very interesting and difficult problem. The bulk of the work is finally done and is running on the AI test server.
NPC countries have the ability to recall goods, recall tech, and perform market history searches. It was fairly straightforward to add this ability. The tough part is making the NPCs actually use it well.
I don't think that there's a clear answer as to the best way for a country to pick what military goods or tech that it produces. Right now, I feel that the NPC countries are too sensitive to current market prices and can get tricked into doing unhelpful things as a result. For example, if there's been no med tech sales during the set then why bother producing it? However, make things too optimal can cause issues as well. Generally speaking, a techer has the strongest incentive to produce the techs that sell for the most, which are bus, res, agri, and indy. If all of the NPCs only teched those techs then players wouldn't be able to buy mil tech, mil strat, and weapons. So there does need to be some kind of balance. To try to get to that balance, I've programmed five different production algorithms for techers and indies. A country picks an algorithm at the start of the set. Current choices:
1) Produce goods based on current market prices
2) Produce goods based on average market prices over some time period
3) Produce goods based on high market prices over some time period
4) Produce goods based on total sales over some time period
5) Produce goods randomly
Obviously some of these are better than others, but I think it's important to have different countries doing different things. Here's an example of an indy picking production values based on historical sales:
[19:55:56] Getting military prices using market search
[19:55:56] Market history for m_tr: {"low_price":"66","high_price":"143","total_units_sold":"5366569","total_sales":"576284839","avg_price":107.38,"no_results":false}
[19:55:56] Market history for m_j: no data
[19:55:56] Market history for m_tu: {"low_price":"51","high_price":"195","total_units_sold":"26210664","total_sales":"3405673791","avg_price":129.93,"no_results":false}
[19:55:56] Market history for m_ta: {"low_price":"358","high_price":"554","total_units_sold":"994135","total_sales":"486696948","avg_price":489.57,"no_results":false}
[19:55:56] Setting indy production using algorithm SALES
[19:55:56] Unit score for m_tr is 1071889801 with value 576284839, production 1.86, and default 10
[19:55:56] Unit score for m_j is 112 with value 60, production 1.86, and default 60
[19:55:56] Unit score for m_tu is 6334553251 with value 3405673791, production 1.86, and default 200
[19:55:56] Unit score for m_ta is 194678779 with value 486696948, production 0.4, and default 40
[19:55:56] Normalizing indy production scores so they sum to 100
[19:55:56] --- Indy production: 1% pro_spy 13% pro_tr 0% pro_j 84% pro_tu 2% pro_ta
Previously, that indy would have set production to 99% for the unit that had the highest current market price.
Here's an example of a techer setting production based on historical high prices:
[20:04:43] Getting tech prices using market search
[20:04:43] Market history for t_mil: {"low_price":"901","high_price":"7638","total_units_sold":"209155","total_sales":"300438668","avg_price":1436.44,"no_results":false}
[20:04:43] Market history for t_med: no data
[20:04:43] Market history for t_bus: {"low_price":"1293","high_price":"5115","total_units_sold":"2961493","total_sales":"8269923111","avg_price":2792.48,"no_results":false}
[20:04:43] Market history for t_res: {"low_price":"1617","high_price":"5376","total_units_sold":"3783894","total_sales":"10491088117","avg_price":2772.56,"no_results":false}
[20:04:43] Market history for t_agri: {"low_price":"906","high_price":"9573","total_units_sold":"698140","total_sales":"2278617635","avg_price":3263.84,"no_results":false}
[20:04:43] Market history for t_war: no data
[20:04:43] Market history for t_ms: no data
[20:04:43] Market history for t_weap: {"low_price":"906","high_price":"1175","total_units_sold":"77668","total_sales":"77019962","avg_price":991.66,"no_results":false}
[20:04:43] Market history for t_indy: {"low_price":"909","high_price":"2729","total_units_sold":"890521","total_sales":"1329257264","avg_price":1492.67,"no_results":false}
[20:04:43] Market history for t_spy: no data
[20:04:43] Market history for t_sdi: no data
[20:04:43] Setting tech production using algorithm HIGH_PRICE
[20:04:43] Unit shares in order of price ascending: [2,2,2,2,4,4,4,15,15,25,25]
[20:04:43] Initial unit score for t_mil is 7638 with price 7638 and production 1
[20:04:43] Initial unit score for t_med is 0 with price 0 and production 1
[20:04:43] Initial unit score for t_bus is 5115 with price 5115 and production 1
[20:04:43] Initial unit score for t_res is 5376 with price 5376 and production 1
[20:04:43] Initial unit score for t_agri is 9573 with price 9573 and production 1
[20:04:43] Initial unit score for t_war is 0 with price 0 and production 1
[20:04:43] Initial unit score for t_ms is 0 with price 0 and production 1
[20:04:43] Initial unit score for t_weap is 1175 with price 1175 and production 1
[20:04:43] Initial unit score for t_indy is 2729 with price 2729 and production 1
[20:04:43] Initial unit score for t_spy is 0 with price 0 and production 1
[20:04:43] Initial unit score for t_sdi is 0 with price 0 and production 1
[20:04:43] Normalizing production scores to sum to 10000 for easier reading
[20:04:43]
Tech production scores:
{"t_agri":2500,"t_mil":2500,"t_res":1500,"t_bus":1500,"t_indy":400,"t_weap":400,"t_med":400,"t_war":200,"t_ms":200,"t_spy":200,"t_sdi":200}
Based on high prices, it decided to put 25% of its tpt into agri, 25% into mil, 15% into bus and res, and the rest of techs get 2 or 4%.
For selling prices, I've tightened up the allowed range for price swings and given farmers, indies, and techers the optional to price their goods based on average prices instead of current price.
Countries will have the ability to stockpile if they get too much cash on hand. I expect this to be of limited use on Express and to be of greater importance for FFA and Alliance. Perhaps once the Express cashers and farmers improve more they will be able to stockpile here.
Selling has been improved during destocking to be more marketed based as well. Countries will continue to sell tech and bushels at market prices while they destock. One problem with the current approach is that techers dump at minimum prices once they start destocking, even if current prices are high. That's been fixed. I was able to make this better because countries can recall goods and tech now. So a techer can try to sell tech at higher prices, and close to the end of the set it will give up, recall tech, and dump at minimum prices then.
As I said earlier, selling algorithms and prices are what I consider to be a difficult problem to solve - probably the most difficult out of anything that the NPCs can do. It will probably require some testing iterations on servers with players to get it into a good state. On its own, I don't think that these changes will be able to get the tech market into a more ideal state. Here's what I think is happening today:
* NPC techers tech too much too early and try to sell a lot of tech.
* NPN non-techers don't have tuned startups so they take a while before they start buying tech.
So you end up with a pattern of lower tech prices at the start with them rising throughout. I do want to see early techers teching less and non-techers buying more tech early on, but that won't be part of the current set of changes that's in progress. I'm also not sure if Express will ever see the tech price pattern that's typical on other servers. Presumably, human techers want to play their turns early in order to maximize their opportunity to sell tech. However, human cashers can save up to 40% of the set's turn count while waiting for their preferred tech prices. So there might fundamentally be a mismatch there that can't (and maybe shouldn't) be resolved with NPC updates. With all of that said, techer is still overpowered on Express so I'm not too worried about it at this time.
Feel free to ask questions about the NPCs!
1) Better destocking
2) Better NPC logging and reporting
3) Better public market purchasing
Going back to the other post, these items are remaining:
4) Better public market selling
5) Misc issue clean up
6) Add retals for bots
This post is about development that's still in progress, so the details might change. Much of my focus lately has been on improving public market selling, which is a very interesting and difficult problem. The bulk of the work is finally done and is running on the AI test server.
NPC countries have the ability to recall goods, recall tech, and perform market history searches. It was fairly straightforward to add this ability. The tough part is making the NPCs actually use it well.
I don't think that there's a clear answer as to the best way for a country to pick what military goods or tech that it produces. Right now, I feel that the NPC countries are too sensitive to current market prices and can get tricked into doing unhelpful things as a result. For example, if there's been no med tech sales during the set then why bother producing it? However, make things too optimal can cause issues as well. Generally speaking, a techer has the strongest incentive to produce the techs that sell for the most, which are bus, res, agri, and indy. If all of the NPCs only teched those techs then players wouldn't be able to buy mil tech, mil strat, and weapons. So there does need to be some kind of balance. To try to get to that balance, I've programmed five different production algorithms for techers and indies. A country picks an algorithm at the start of the set. Current choices:
1) Produce goods based on current market prices
2) Produce goods based on average market prices over some time period
3) Produce goods based on high market prices over some time period
4) Produce goods based on total sales over some time period
5) Produce goods randomly
Obviously some of these are better than others, but I think it's important to have different countries doing different things. Here's an example of an indy picking production values based on historical sales:
[19:55:56] Getting military prices using market search
[19:55:56] Market history for m_tr: {"low_price":"66","high_price":"143","total_units_sold":"5366569","total_sales":"576284839","avg_price":107.38,"no_results":false}
[19:55:56] Market history for m_j: no data
[19:55:56] Market history for m_tu: {"low_price":"51","high_price":"195","total_units_sold":"26210664","total_sales":"3405673791","avg_price":129.93,"no_results":false}
[19:55:56] Market history for m_ta: {"low_price":"358","high_price":"554","total_units_sold":"994135","total_sales":"486696948","avg_price":489.57,"no_results":false}
[19:55:56] Setting indy production using algorithm SALES
[19:55:56] Unit score for m_tr is 1071889801 with value 576284839, production 1.86, and default 10
[19:55:56] Unit score for m_j is 112 with value 60, production 1.86, and default 60
[19:55:56] Unit score for m_tu is 6334553251 with value 3405673791, production 1.86, and default 200
[19:55:56] Unit score for m_ta is 194678779 with value 486696948, production 0.4, and default 40
[19:55:56] Normalizing indy production scores so they sum to 100
[19:55:56] --- Indy production: 1% pro_spy 13% pro_tr 0% pro_j 84% pro_tu 2% pro_ta
Previously, that indy would have set production to 99% for the unit that had the highest current market price.
Here's an example of a techer setting production based on historical high prices:
[20:04:43] Getting tech prices using market search
[20:04:43] Market history for t_mil: {"low_price":"901","high_price":"7638","total_units_sold":"209155","total_sales":"300438668","avg_price":1436.44,"no_results":false}
[20:04:43] Market history for t_med: no data
[20:04:43] Market history for t_bus: {"low_price":"1293","high_price":"5115","total_units_sold":"2961493","total_sales":"8269923111","avg_price":2792.48,"no_results":false}
[20:04:43] Market history for t_res: {"low_price":"1617","high_price":"5376","total_units_sold":"3783894","total_sales":"10491088117","avg_price":2772.56,"no_results":false}
[20:04:43] Market history for t_agri: {"low_price":"906","high_price":"9573","total_units_sold":"698140","total_sales":"2278617635","avg_price":3263.84,"no_results":false}
[20:04:43] Market history for t_war: no data
[20:04:43] Market history for t_ms: no data
[20:04:43] Market history for t_weap: {"low_price":"906","high_price":"1175","total_units_sold":"77668","total_sales":"77019962","avg_price":991.66,"no_results":false}
[20:04:43] Market history for t_indy: {"low_price":"909","high_price":"2729","total_units_sold":"890521","total_sales":"1329257264","avg_price":1492.67,"no_results":false}
[20:04:43] Market history for t_spy: no data
[20:04:43] Market history for t_sdi: no data
[20:04:43] Setting tech production using algorithm HIGH_PRICE
[20:04:43] Unit shares in order of price ascending: [2,2,2,2,4,4,4,15,15,25,25]
[20:04:43] Initial unit score for t_mil is 7638 with price 7638 and production 1
[20:04:43] Initial unit score for t_med is 0 with price 0 and production 1
[20:04:43] Initial unit score for t_bus is 5115 with price 5115 and production 1
[20:04:43] Initial unit score for t_res is 5376 with price 5376 and production 1
[20:04:43] Initial unit score for t_agri is 9573 with price 9573 and production 1
[20:04:43] Initial unit score for t_war is 0 with price 0 and production 1
[20:04:43] Initial unit score for t_ms is 0 with price 0 and production 1
[20:04:43] Initial unit score for t_weap is 1175 with price 1175 and production 1
[20:04:43] Initial unit score for t_indy is 2729 with price 2729 and production 1
[20:04:43] Initial unit score for t_spy is 0 with price 0 and production 1
[20:04:43] Initial unit score for t_sdi is 0 with price 0 and production 1
[20:04:43] Normalizing production scores to sum to 10000 for easier reading
[20:04:43]
Tech production scores:
{"t_agri":2500,"t_mil":2500,"t_res":1500,"t_bus":1500,"t_indy":400,"t_weap":400,"t_med":400,"t_war":200,"t_ms":200,"t_spy":200,"t_sdi":200}
Based on high prices, it decided to put 25% of its tpt into agri, 25% into mil, 15% into bus and res, and the rest of techs get 2 or 4%.
For selling prices, I've tightened up the allowed range for price swings and given farmers, indies, and techers the optional to price their goods based on average prices instead of current price.
Countries will have the ability to stockpile if they get too much cash on hand. I expect this to be of limited use on Express and to be of greater importance for FFA and Alliance. Perhaps once the Express cashers and farmers improve more they will be able to stockpile here.
Selling has been improved during destocking to be more marketed based as well. Countries will continue to sell tech and bushels at market prices while they destock. One problem with the current approach is that techers dump at minimum prices once they start destocking, even if current prices are high. That's been fixed. I was able to make this better because countries can recall goods and tech now. So a techer can try to sell tech at higher prices, and close to the end of the set it will give up, recall tech, and dump at minimum prices then.
As I said earlier, selling algorithms and prices are what I consider to be a difficult problem to solve - probably the most difficult out of anything that the NPCs can do. It will probably require some testing iterations on servers with players to get it into a good state. On its own, I don't think that these changes will be able to get the tech market into a more ideal state. Here's what I think is happening today:
* NPC techers tech too much too early and try to sell a lot of tech.
* NPN non-techers don't have tuned startups so they take a while before they start buying tech.
So you end up with a pattern of lower tech prices at the start with them rising throughout. I do want to see early techers teching less and non-techers buying more tech early on, but that won't be part of the current set of changes that's in progress. I'm also not sure if Express will ever see the tech price pattern that's typical on other servers. Presumably, human techers want to play their turns early in order to maximize their opportunity to sell tech. However, human cashers can save up to 40% of the set's turn count while waiting for their preferred tech prices. So there might fundamentally be a mismatch there that can't (and maybe shouldn't) be resolved with NPC updates. With all of that said, techer is still overpowered on Express so I'm not too worried about it at this time.
Feel free to ask questions about the NPCs!