FM Data Export Visualisations

[JPKD]

Been playing around with the data export option in FM20/21. Until now I had no idea that if you select "Print Screen" from the FM menu dropdown you can then export data in HTML format to copy/paste straight in to Excel or Google Sheets.

On the squad screen you can create a view with every Chalkboard stat and pull it out of the game. This can even be done for opposition clubs. So my 4am mind was whirring earlier and I was thinking you could export club team data to Google Sheets and set up a Data Studio scounting report based on all the teams stats. 

So Man City's player data would go from this:

 

To this:

 

 

While I admit the presentation is a little rough around the edges I think you can get the idea. This might be even more useful when the game is full of regens and you need to quickly identify goalscoring or creative threats (not 100% sure how quick you can refresh the data right now).

You can even export around 260 players at a time from the Scouting Search function to create datasets on entire divisions.

 

 

Is anyone interested in seeing how far I could go with this? I could potentially set up a template for people to copy and fill with their own data (a less than 2 minute task) to get a view on their own teams/games. 

Equally has anyone got a good source of football visualisations that they think would work with FM data - please no Radar charts at this stage. 

[saihtam]

This looks really amazing. I would love to get a bit more detailed instructions how you achive this view and I am sure I wont be alone.

Maybe in the future also post in suggestion forum as they have this info, they can easily visualise this in the game also.

Very good find.

[JPKD]

Yeah I'm just playing around at the moment. Once I've built up a few template I'll wirte up a full guide on how to export and ideas for creating new charts. Not quite sure where it should sit in the forum.

A full blown visuliser tool inside the game would be even better. I feel like there is loads of data in game but some of it is really hard to access. I've seen people asking for xG per player across more than one game. It is in there but you have to go to the Player > Reports > Coach Report and open the Radar chart to get the number. Even then it's per 90 minutes and not available for defenders. Here's hoping we'll be able to get xG on the squad screen this year that would really complete the Goals chart above. 

Edited November 18, 2020 by JPKD

[Mikal]

At the back end of FM20 I turned attributes off and played this way (using statistics only). Posted some up in the careers forum, here's a link to one of the posts for example: 

I started playing this way in the middle of a save in 2030 when the game was full of newgens. I used it largely for scouting, squad comparison and tracking team/individual performance.

One of the commenters on that thread ( @cmason84) was also playing this way and posting on his blog (very interesting reading from what you can do with the data in FM).

FWIW, you can also extract much more than 260 from the scouting page, I routinely extract thousands of lines. I did a test already and I think I extracted around 50,000 (almost the entire player database) but it absolutely did a number on my laptop lol.

[JPKD]

On 18/11/2020 at 08:38, Mikal said:

FWIW, you can also extract much more than 260 from the scouting page, I routinely extract thousands of lines. I did a test already and I think I extracted around 50,000 (almost the entire player database) but it absolutely did a number on my laptop lol.

Thanks, interesting stuff. How did you do the larger export?

Been playing about with how to structure the data with an eye on making it a template and visual elements to match the default FM21 skin:

[hardayas]

@Mikal

 

interested in knowing how to get a larger export as well.

 

 

[gam945]

@JPKD Are you familiar with R and SQL? I've wanted a lot of times to this sort of analysis and most of the time I abandoned cause even if I could produce those graphs, I had no platform to regroup the graphs and be able to do my analysis... Now at work I'm working on a project using Shiny from RStudio, and it offers exactly what I was looking for before  I'm planning to build a web app that enables us to manage our club using data, from salaries to squad analysis to scouting. The beauty of Shiny is that you need no experience with classic web dev tools such as html and css: it's all wrapped in R code 

Will surely create a thread later about it

Edited November 19, 2020 by gam945

[saihtam]

10 hours ago, JPKD said:

Thanks, interesting stuff. How did you do the larger export?

Been playing about with how to structure the data with an eye on making it a template and visual elements to match the default FM21 skin:

How long one graphc making takes time for you? I mean from exporting data out of FM and getting the graph

[hardayas]

@gam945

Quote

Are you familiar with R and SQL? I've wanted a lot of times to this sort of analysis and most of the time I abandoned cause even if I could produce those graphs, I had no platform to regroup the graphs and be able to do my analysis... Now at work I'm working on a project using Shiny from RStudio, and it offers exactly what I was looking for before  I'm planning to build a web app that enables us to manage our club using data, from salaries to squad analysis to scouting. The beauty of Shiny is that you need no experience with classic web dev tools such as html and css: it's all wrapped in R code 

Will surely create a thread later about it

I would be really interested in that. It sounds like a data analyst's and a fm enthusiast's pipe dream.

[JPKD]

3 hours ago, gam945 said:

@JPKD Are you familiar with R and SQL? I've wanted a lot of times to this sort of analysis and most of the time I abandoned cause even if I could produce those graphs, I had no platform to regroup the graphs and be able to do my analysis... Now at work I'm working on a project using Shiny from RStudio, and it offers exactly what I was looking for before  I'm planning to build a web app that enables us to manage our club using data, from salaries to squad analysis to scouting. The beauty of Shiny is that you need no experience with classic web dev tools such as html and css: it's all wrapped in R code 

Will surely create a thread later about it

I'd say I understand how SQL works, I've never done any database building but I've done a bit of query writing but I'm far from an everday user. Anything I've built so far is just dumped in to a Google Sheet and run through Data Studio.

58 minutes ago, saihtam said:

How long one graphc making takes time for you? I mean from exporting data out of FM and getting the graph

Probably about 15 minutes to do a rough version. Problem is each time you pull out different stats you break the graphics. So I'm trying to work out every stat I'll need to include in the template. Data Studio has a really nice theme builder that automatically matched the colours from an image of the FM21 logo.

[saihtam]

3 hours ago, gam945 said:

@JPKD Are you familiar with R and SQL? I've wanted a lot of times to this sort of analysis and most of the time I abandoned cause even if I could produce those graphs, I had no platform to regroup the graphs and be able to do my analysis... Now at work I'm working on a project using Shiny from RStudio, and it offers exactly what I was looking for before  I'm planning to build a web app that enables us to manage our club using data, from salaries to squad analysis to scouting. The beauty of Shiny is that you need no experience with classic web dev tools such as html and css: it's all wrapped in R code 

Will surely create a thread later about it

If you look people to do this with can PM me also. Dont have huge background on SQL writing, know the basic etc. Been QA ( + learning master in Software architecture currently)  for 5 years so have my share of connection with it.

Already had vision to make this thing work faster to put somekind of automation in. When the data is exported to specific folder, the background program would do the work to quickly visualise it. So you could in the game export needed info and later you can quickly check the results

[gam945]

1 minute ago, saihtam said:

If you look people to do this with can PM me also. Dont have huge background on SQL writing, know the basic etc. Been QA ( + learning master in Software architecture currently)  for 5 years so have my share of connection with it.

Already had vision to make this thing work faster to put somekind of automation in. When the data is exported to specific folder, the background program would do the work to quickly visualise it. So you could in the game export needed info and later you can quickly check the results

will definitively PM you and @hardayas. I'm myself not an expert, but know the basics+ such as maintaining it and extracting data from it. R however I'd say intermediary+, used it (and still using) in school and at work, but not at data scientist level. I've already installed PostgreSQL (initially wanted to use SQL server express, the free version, but there's limitations on the DB size, and Access won't be good as there will be heavy querying from the R Shiny web app to the DB). Will PM soon, after I finalize a suggestion of how the app will be like and so then we can either discuss it or approve and begin to create views on FM to update the database.

[JPKD]

On 19/11/2020 at 11:16, hardayas said:

@Mikal

 

interested in knowing how to get a larger export as well.

 

 

Found out how you do this. When on the Player Search screen use Ctrl + A to select every player in the search, then export the file and every player in the search is exported.

[hardayas]

thanks @JPKD. works like a charm

[cmason84]

On 27/11/2020 at 07:46, JPKD said:

Found out how you do this. When on the Player Search screen use Ctrl + A to select every player in the search, then export the file and every player in the search is exported.

Interesting, I didn't realise this but I'd largely only been exporting players in a specific position with 1,000+ minutes in certain divisions so wasn't ever exporting that many players. Number of minutes is important to avoid 'noise'.

For those interested in this area, you'll probably find my FM20 blog interesting so let me know if you want links to the blog site. I'll be doing something similar again this year but I've not decided upon my save yet.

[cmason84]

I found making these quite good fun, so it's good to see others using the data from FM. I did all this just using Excel and PowerPoint so didn't need to understand R (which I've zero idea of).

[JPKD]

9 hours ago, cmason84 said:

I found making these quite good fun, so it's good to see others using the data from FM. I did all this just using Excel and PowerPoint so didn't need to understand R (which I've zero idea of).

 

Percentile rank is a really nive way to understand a player. I wish I could do more with this but work and family are taking up more of my time than ever, to the extent tht for the time being I'm prioritising playing the game over doing some of this. In previous years I woul have a couple of hours a day on the train to mess about with this. Part of the problem is I'm too much of a perfectionist so I want to do it properly. Will try and get an evening or a day at Christmas to nail some fundamentals and get a bit of a guide together.

 

My biggest challenge is the "so what?" question that anyone should be asking before embarking on data visualisation. Basically what do I/we want levers in our tactics to we want to pull as a resukt of looking in to this? Then I can reverse enginerr some useful stats and visualisations.

[cmason84]

10 hours ago, JPKD said:

 

Percentile rank is a really nive way to understand a player. I wish I could do more with this but work and family are taking up more of my time than ever, to the extent tht for the time being I'm prioritising playing the game over doing some of this. In previous years I woul have a couple of hours a day on the train to mess about with this. Part of the problem is I'm too much of a perfectionist so I want to do it properly. Will try and get an evening or a day at Christmas to nail some fundamentals and get a bit of a guide together.

 

My biggest challenge is the "so what?" question that anyone should be asking before embarking on data visualisation. Basically what do I/we want levers in our tactics to we want to pull as a resukt of looking in to this? Then I can reverse enginerr some useful stats and visualisations.

If you want to do a join piece on this, let me know - I'd be happy to help with the guide. I'd even host it on my blog if you'd be interested?

I'm prioritising work and my gf over playing FM at all at the moment. I've barely scratched the surface with FM21 so far but by enlarge I'm impressed with it given the situation.

[CowShedBarmyArmy]

Hi all,

Got a weird fault and was wondering if someone could help. When I export players from player search or shortlist I lose most of the names. They get replaced by --

Squad export does the same. I've used a few other filters and the same thing happens.

[canadice]

On 03/08/2022 at 20:23, CowShedBarmyArmy said:

Hi all,

Got a weird fault and was wondering if someone could help. When I export players from player search or shortlist I lose most of the names. They get replaced by --

Squad export does the same. I've used a few other filters and the same thing happens.

For some reason the export/print function only prints data that "it has seen", so a quick fix for this is to scroll through the entire list you want to export from start to finish. Make sure to allow all rows to be seen. I tend to use the mouse wheel to not make the scrolling go too fast.

[_Ben_]

On 01/12/2020 at 22:27, cmason84 said:

I found making these quite good fun, so it's good to see others using the data from FM. I did all this just using Excel and PowerPoint so didn't need to understand R (which I've zero idea of).

I was so impressed by your blogs a few years back that I just decided to build them inside FM: