Popular Post CowShedBarmyArmy Posted May 5, 2020 Popular Post Share Posted May 5, 2020 (edited) Analysing FM with Python I know MS Excel has become very popular on here for analysing squads etc but does anyone use Python? Its very powerful and once you have the code setup you can create charts and tables directly from FM20 without any manual entry required. I've been learning it while working from home during the pandemic and find it easier to use than excel for squad analysis and analysing scouting targets. What it does: I've been using it to rate my squad by DNA score and position. For each position in my current tactic I have a filtered table and scatter plot. I use these to assess my current squad and players found in the player search to see who would fit in to my style. I also analyse Club DNA Here is an example of the code to show how easy it is to edit. How to use it: I created two squad views: One with all attributes One with all stats. I also create the same view in the scouting module so I can assess players from my searches. I still need to create my stats one. I export these (ctrl-p --- print to webpage) to a nominated folder on my HDD with a fixed file name. (1) Install anaconda which is open source and free for single home users: https://www.anaconda.com/products/individual This contains Jupyter Notebooks which I used to enter my code. (2) You will need to install the following packages numpy, pandas, plotly, cufflinks Here is how to do that: https://docs.anaconda.com/anaconda/user-guide/tasks/install-packages/ Edit the paths to the html files from FM20 Once I run my file in Jupyter, It imports all the data from the .html files and populates my tables and graphs automatically. It takes a few seconds and I have lots of data to play with. I'm still pretty basic at the moment but happy to share code etc with anyone or take advice from anyone who knows what they are doing. Edited May 10, 2020 by CowShedBarmyArmy Adding info 14 Link to post Share on other sites More sharing options...
Guest Posted May 5, 2020 Share Posted May 5, 2020 Nice staff here! Link to post Share on other sites More sharing options...
Guest Posted May 5, 2020 Share Posted May 5, 2020 @Giggiux interested to see that @CowShedBarmyArmy Link to post Share on other sites More sharing options...
gam945 Posted May 6, 2020 Share Posted May 6, 2020 Always wanted to do something similar and doing the "analysis" job of the manager IRL, for example monitoring the squad during the season with stats and attributes...but laziness always takes over Planning of beginning a new long term save soon, I'll definitively try to be serious about it lol Also this could be a very useful tool for scouting hundreds of players stats Will share here if I get something done Link to post Share on other sites More sharing options...
CowShedBarmyArmy Posted May 6, 2020 Author Share Posted May 6, 2020 I'm happy to share my code and view etc to get you started if you want. This is just for fun for me, not looking to profit. 4 Link to post Share on other sites More sharing options...
22764636 Posted May 6, 2020 Share Posted May 6, 2020 I have compiled a python script that tells me what's the best role-duty for a player given his attributes 1 Link to post Share on other sites More sharing options...
CowShedBarmyArmy Posted May 6, 2020 Author Share Posted May 6, 2020 12 minutes ago, 22764636 said: I have compiled a python script that tells me what's the best role-duty for a player given his attributes Nice, one. Good to know there's another nerd out there! Feel free to share your stuff on here. Link to post Share on other sites More sharing options...
PenguinKenny Posted May 6, 2020 Share Posted May 6, 2020 2 hours ago, 22764636 said: I have compiled a python script that tells me what's the best role-duty for a player given his attributes I'd be interested in seeing this. Link to post Share on other sites More sharing options...
CowShedBarmyArmy Posted May 9, 2020 Author Share Posted May 9, 2020 Here is a link to the code if anyone is interested. https://github.com/winston1984-sys/FM20-Analysis 3 Link to post Share on other sites More sharing options...
Egecann Posted May 10, 2020 Share Posted May 10, 2020 I’m gonads definitely learn python during quarantine. Thanks for the code btw Link to post Share on other sites More sharing options...
CowShedBarmyArmy Posted May 10, 2020 Author Share Posted May 10, 2020 6 hours ago, Egecann said: I’m gonads definitely learn python during quarantine. Thanks for the code btw No worries, give me a shout if you need any help. Link to post Share on other sites More sharing options...
gam945 Posted May 10, 2020 Share Posted May 10, 2020 (edited) @CowShedBarmyArmy wondering how do you visualize your plots? EDIT: Will just go with Jupyter Lab/Notebook, less of a hassle and it would be sort of a report I have defined all the plots and data I wanna see using my excel tool (which I can share): It contains a dictionnary for the attributes (long name, short name), the stats (long & short name, categorization also) a sheet to define my tactic and a sheet with a macro that converts multiple cells into a string containing their values into a python list format, for example ['aaa','bbb','ccc'] I've already wrote a code to import data and clean it, for example converting appropriate stats to integer of float type, or converting appearances (x) y into x+y. An example of simple plot with matplotlib: In the long term, I also project to analyse how attributes influence my KPIs, so projecting to do some regression analysis after setting up my dashboard, would be a cool project Edited May 10, 2020 by gam945 1 Link to post Share on other sites More sharing options...
CowShedBarmyArmy Posted May 11, 2020 Author Share Posted May 11, 2020 19 hours ago, gam945 said: @CowShedBarmyArmy wondering how do you visualize your plots? EDIT: Will just go with Jupyter Lab/Notebook, less of a hassle and it would be sort of a report I have defined all the plots and data I wanna see using my excel tool (which I can share): It contains a dictionnary for the attributes (long name, short name), the stats (long & short name, categorization also) a sheet to define my tactic and a sheet with a macro that converts multiple cells into a string containing their values into a python list format, for example ['aaa','bbb','ccc'] I've already wrote a code to import data and clean it, for example converting appropriate stats to integer of float type, or converting appearances (x) y into x+y. An example of simple plot with matplotlib: In the long term, I also project to analyse how attributes influence my KPIs, so projecting to do some regression analysis after setting up my dashboard, would be a cool project Hi gam945, I use Plotly at the moment, couldnt quite work out how to add the player name to a scatter plot on matplotlib. https://plotly.com/python/plotly-express/ Jupyter is very easy to use so I would definately recommend it. It allows you to test small chunks of code at a time. I'm impressed with the clarity of your plan. I planned to create this kind of detailed plan up front but I just ending up diving straight in and I've been winging it ever since. I cant wait to see your progress. 1 Link to post Share on other sites More sharing options...
WhoAreYa? Posted May 12, 2020 Share Posted May 12, 2020 (edited) This is fantastic! The Python aspect is perhaps beyond me but your statistical analysis is very much up my street. I've just started a save with Brentford FC who are massive on stats in real life and it is definitely an area that I'm going to be placing a lot of emphasis on, in the transfer market especially. Football seems to be becoming more analytical by the day and, honestly, it surprises me that more people don't look at the in-game stats which are actually quite numerous now - and will only become more so. There are some that I find very important such as save % for keepers, shots on target for strikers and key passes. What are your favourite stats and how do these change by position or by role? For example what stats do you place most emphasis on for say central defenders? Edited May 12, 2020 by WhoAreYa? Link to post Share on other sites More sharing options...
CowShedBarmyArmy Posted May 12, 2020 Author Share Posted May 12, 2020 (edited) Thanks for the nice feedback. Maybe one day I will be competent enough to make the program really easy to use! I haven't finalised my stats analysis yet as I have been developing the python code but for central defenders I am currently using : Interceptions/90 vs Headers won/90 I plan to use the stats in here https://totalfootballanalysis.com/category/data-analysis which will mean I have to create some custom ones. Work has been busy today so havent done much last two days but I will soon. Edited May 12, 2020 by CowShedBarmyArmy Link to post Share on other sites More sharing options...
WhoAreYa? Posted May 13, 2020 Share Posted May 13, 2020 This may be of interest to you, @CowShedBarmyArmy? https://theresonlyoneball.com/2019/12/31/0104-scouting-and-transfers/ from @fmFutbolManager As I mentioned with my Brentford save, this is the kind of thing that I want to be setting up. Fascinating stuff! Link to post Share on other sites More sharing options...
FMFutbol Posted May 14, 2020 Share Posted May 14, 2020 Will try this later! Link to post Share on other sites More sharing options...
CowShedBarmyArmy Posted May 14, 2020 Author Share Posted May 14, 2020 (edited) 21 hours ago, WhoAreYa? said: This may be of interest to you, @CowShedBarmyArmy? https://theresonlyoneball.com/2019/12/31/0104-scouting-and-transfers/ from @fmFutbolManager As I mentioned with my Brentford save, this is the kind of thing that I want to be setting up. Fascinating stuff! Thanks for the link, I've read his posts before and they are excellent. I will try and integrate some ideas into my stuff. I've made some progress today with converting data etc so it should be full steam ahead now. Think I have a decent way to assess goalkeepers: Saves/90 vs Goals Conceded/90 Theory being a keeper making lots of saves in a team that doesnt concede much must be doing ok. I'd love total shots faced but I can't find that in FM20. I had to calculate Saves/90 myself as it doesn't exist. If you have specific metrics you'd like then let me know and i'll create them and upload the file. Edited May 14, 2020 by CowShedBarmyArmy Link to post Share on other sites More sharing options...
CowShedBarmyArmy Posted May 14, 2020 Author Share Posted May 14, 2020 17 hours ago, FMFutbol said: Will try this later! If you need help give me a shout. I'm slowly going mad creating this stuff but I'm 6 games from an unbeaten season with 1860 Munich so it must be working Link to post Share on other sites More sharing options...
CowShedBarmyArmy Posted May 14, 2020 Author Share Posted May 14, 2020 On 10/05/2020 at 01:15, Egecann said: I’m gonads definitely learn python during quarantine. Thanks for the code btw I'm uploading new versions all the time so keep checking! Link to post Share on other sites More sharing options...
WhoAreYa? Posted May 14, 2020 Share Posted May 14, 2020 1 hour ago, CowShedBarmyArmy said: Thanks for the link, I've read his posts before and they are excellent. I will try and integrate some ideas into my stuff. I've made some progress today with converting data etc so it should be full steam ahead now. Think I have a decent way to assess goalkeepers: Saves/90 vs Goals Conceded/90 Theory being a keeper making lots of saves in a team that doesnt concede much must be doing ok. I'd love total shots faced but I can't find that in FM20. I had to calculate Saves/90 myself as it doesn't exist. If you have specific metrics you'd like then let me know and i'll create them and upload the file. I seem to remember that there's a stat shown on a player's data profile that isn't shown on the search filter. Their personal data report card near where you look at their coach report. Sure it's something like shots faced or saves made. I can't check right now as I'm waiting for a new computer and can't quite remember it! Link to post Share on other sites More sharing options...
CowShedBarmyArmy Posted May 15, 2020 Author Share Posted May 15, 2020 Two new files uploaded to GitHub. Added message box to choose between Current Squad data or Potential Signings. This saves having too much data on screen at one time. Now the coding is in reasonable shape (for a rookie), I'll focus on creating new player profile scatter plots. At the moment I only have: Goalkeepers - Saves/90 v Goals Conceded/90 Best Tacklers - Tackles/90 v Tackle Won % Best Headers - Headers Won/90 v Header Win % Ball Winners - Tackles/90 v Interceptions/90 Wing Play - Dribbles Per Game v Assists/90 Playmakers - Key Passes/90 v Chances Created/90 Goalscorers - Goals/90 v Shot On Target % Link to post Share on other sites More sharing options...
CowShedBarmyArmy Posted May 19, 2020 Author Share Posted May 19, 2020 (edited) On 14/05/2020 at 20:50, WhoAreYa? said: I seem to remember that there's a stat shown on a player's data profile that isn't shown on the search filter. Their personal data report card near where you look at their coach report. Sure it's something like shots faced or saves made. I can't check right now as I'm waiting for a new computer and can't quite remember it! Think i've sussed it out. shots saved + goals conceded = shots faced. Cheers for the heads up Edited May 19, 2020 by CowShedBarmyArmy Link to post Share on other sites More sharing options...
nick1408 Posted May 23, 2020 Share Posted May 23, 2020 Any chance you could work out xG with this? I'm quite interested to try it out https://dictatethegame.com/2019/09/27/getting-expected-goals-or-xg-into-fm19/ Link to post Share on other sites More sharing options...
CowShedBarmyArmy Posted May 23, 2020 Author Share Posted May 23, 2020 Apart from method one they require manual calculations so I don't think I can achieve this in python. I can try the quick and dirty one though. Link to post Share on other sites More sharing options...
Guest Posted May 23, 2020 Share Posted May 23, 2020 9 hours ago, nick1408 said: Any chance you could work out xG with this? I'm quite interested to try it out https://dictatethegame.com/2019/09/27/getting-expected-goals-or-xg-into-fm19/ This must come with the game IMHO. Link to post Share on other sites More sharing options...
FMFutbol Posted May 27, 2020 Share Posted May 27, 2020 On 15/05/2020 at 17:47, CowShedBarmyArmy said: Two new files uploaded to GitHub. Added message box to choose between Current Squad data or Potential Signings. This saves having too much data on screen at one time. Now the coding is in reasonable shape (for a rookie), I'll focus on creating new player profile scatter plots. At the moment I only have: Goalkeepers - Saves/90 v Goals Conceded/90 Best Tacklers - Tackles/90 v Tackle Won % Best Headers - Headers Won/90 v Header Win % Ball Winners - Tackles/90 v Interceptions/90 Wing Play - Dribbles Per Game v Assists/90 Playmakers - Key Passes/90 v Chances Created/90 Goalscorers - Goals/90 v Shot On Target % I like the 3d plotly express charts the best as you can include more information. Like for wing play, dribbles per game, cross completion, pass percentage. Feels like I get better information with 3d than just 2d. Looked at trying a 3d surface plot but not sure how to fit it to the data Link to post Share on other sites More sharing options...
CowShedBarmyArmy Posted May 30, 2020 Author Share Posted May 30, 2020 On 27/05/2020 at 23:52, FMFutbol said: I like the 3d plotly express charts the best as you can include more information. Like for wing play, dribbles per game, cross completion, pass percentage. Feels like I get better information with 3d than just 2d. Looked at trying a 3d surface plot but not sure how to fit it to the data I haven't tried these yet, great idea thanks. Link to post Share on other sites More sharing options...
calv84 Posted October 11, 2020 Share Posted October 11, 2020 Hi guys I've been trying to teach myself Python over the last few weeks and fm seemed to be a good place to start, and this thread has been a big inspiration for me and I refer back to it every couple days. My goal is to ultimately unearth good value players from scouting only in game statistics not attributes. An issue that i have run into is converting the Value string into a usable float. The way the value data is extracted from the game makes it incredibly difficult (for my simple mind) to apply a set of rules to convert it into usable data. Hopefully below will highlight the issue i'm running into. The way I had been looking to take the issue is to replace K or M with a set number of 0's and remove the non-numeric characters. This is obviously isn't going to work due to the structure of the exported data. For example if i replaced M with 000000 i would receive: 2m = 2000000 7.5m = 75000000 9.75m = 975000000 Can anyone tell me how they have solved this? Or if there is a different field available? I'm really not keen on this being a manual process. Link to post Share on other sites More sharing options...
andershs Posted October 11, 2020 Share Posted October 11, 2020 (edited) 1 hour ago, calv84 said: Hi guys I've been trying to teach myself Python over the last few weeks and fm seemed to be a good place to start, and this thread has been a big inspiration for me and I refer back to it every couple days. My goal is to ultimately unearth good value players from scouting only in game statistics not attributes. An issue that i have run into is converting the Value string into a usable float. The way the value data is extracted from the game makes it incredibly difficult (for my simple mind) to apply a set of rules to convert it into usable data. Hopefully below will highlight the issue i'm running into. The way I had been looking to take the issue is to replace K or M with a set number of 0's and remove the non-numeric characters. This is obviously isn't going to work due to the structure of the exported data. For example if i replaced M with 000000 i would receive: 2m = 2000000 7.5m = 75000000 9.75m = 975000000 Can anyone tell me how they have solved this? Or if there is a different field available? I'm really not keen on this being a manual process. This is just from the top of my head. But couldn't you do something like this: If you encounter 'm' and you know you are dealing with monetary values: Remove 'm' from the string or trim the string and set the string length to n-1 Try to cast the string to float Multiply the float by 1000000 This can similarly also be applied to 'k' monetary values (multiplying instead with 100000). I know there are some steps and each step could present problems, but it is worth a try. Edited October 11, 2020 by andershs 1 Link to post Share on other sites More sharing options...
calv84 Posted October 11, 2020 Share Posted October 11, 2020 1 hour ago, andershs said: This is just from the top of my head. But couldn't you do something like this: If you encounter 'm' and you know you are dealing with monetary values: Remove 'm' from the string or trim the string and set the string length to n-1 Try to cast the string to float Multiply the float by 1000000 This can similarly also be applied to 'k' monetary values (multiplying instead with 100000). I know there are some steps and each step could present problems, but it is worth a try. Of course... I'll give that a go. Thank you. Link to post Share on other sites More sharing options...
EnigMattic1 Posted October 11, 2020 Share Posted October 11, 2020 (edited) Damn. I really wish I found this thread months ago I actually have the stuff on my laptop to install Python but never got round to it. I kinda wish I had now. Instead, I manually did all the formulas in excel for my club dna AND every position available. 🙄 I always got too confused when it came to some of the stuff like tackles per 90 and tackle win %, so I just didn't bother with that, despite wanting to. Edited October 11, 2020 by EnigMattic1 Link to post Share on other sites More sharing options...
CowShedBarmyArmy Posted November 8, 2020 Author Share Posted November 8, 2020 On 11/10/2020 at 08:42, calv84 said: Hi guys I've been trying to teach myself Python over the last few weeks and fm seemed to be a good place to start, and this thread has been a big inspiration for me and I refer back to it every couple days. My goal is to ultimately unearth good value players from scouting only in game statistics not attributes. An issue that i have run into is converting the Value string into a usable float. The way the value data is extracted from the game makes it incredibly difficult (for my simple mind) to apply a set of rules to convert it into usable data. Hopefully below will highlight the issue i'm running into. The way I had been looking to take the issue is to replace K or M with a set number of 0's and remove the non-numeric characters. This is obviously isn't going to work due to the structure of the exported data. For example if i replaced M with 000000 i would receive: 2m = 2000000 7.5m = 75000000 9.75m = 975000000 Can anyone tell me how they have solved this? Or if there is a different field available? I'm really not keen on this being a manual process. I can send you my code to show you how i solved this. financial = squad[['Name', 'Age', 'Wage', 'Value', 'Expires', 'Mins']].copy() for column in ['Value', 'Wage']: financial[column] = financial[column].str.replace('Â', '') financial[column] = financial[column].str.replace('p/w', '') financial[column] = financial[column].str.replace('£', '').copy() Kind of weird out of context, but hopefully makes sense. Link to post Share on other sites More sharing options...
CowShedBarmyArmy Posted November 8, 2020 Author Share Posted November 8, 2020 On 12/10/2020 at 00:15, EnigMattic1 said: Damn. I really wish I found this thread months ago I actually have the stuff on my laptop to install Python but never got round to it. I kinda wish I had now. Instead, I manually did all the formulas in excel for my club dna AND every position available. 🙄 I always got too confused when it came to some of the stuff like tackles per 90 and tackle win %, so I just didn't bother with that, despite wanting to. If I can help at all give me a shout. I was a complete novice when I started. I still don't fully understand all of what I'm doing but you make big gains by diving in and having a go. Link to post Share on other sites More sharing options...
CowShedBarmyArmy Posted November 8, 2020 Author Share Posted November 8, 2020 (edited) My code is an absolute shambles at the moment as I have been trying all sorts of things and been busy in work but feel free to poke around and have a look. I've uploaded a squad view and scouting view for exporting the data. Also a css file to make the html output look prettier. All you need to do is set the correct path so the program can find your exported fm data (change bold text to your location) squad = pd.read_html( r"C:\Users\stuar\Documents\Sports Interactive\Football Manager 2020\html\fmi_4_sq.html" )[0] scouting = pd.read_html( r"C:\Users\stuar\Documents\Sports Interactive\Football Manager 2020\html\fmi_4_sc.html" )[0] fmi_4.py df_style.css scout_analysis_fmi4.fmf squad_analysis_fmi_4.fmf Edited November 8, 2020 by CowShedBarmyArmy Link to post Share on other sites More sharing options...
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now