-
Posts
186 -
Joined
-
Last visited
Content Type
Forums
Articles
FAQs
Online Manual
Support: Blocks
Support: Games
Bug Tracker
SIGames Manual (beta)
Profiles
Posts posted by gam945
-
-
Fixed the empty data issue! It was coming from currency symbols in my code, replaced them by their unicode characters! Also make sure that you have the following settings before exporting in FM.
32 minutes ago, marculatur said:Preferences - Advanced - Formats
Positive: €1,00 (The Symbol of yout currency has to be before the value)
Decimal Symbol: Dot
Digit Grouping Symbol: Comma
Thanks @marculatur and @IWWROCKS!
If you had already downloaded the app.R file, re-downloaded again from Github and replace the old app.R by it. No need for a full re-install.
2 -
1 minute ago, DarJ said:
Games played is = Minutes played/90
It is more accurate, and you'll see that it's possible to have decimal values.
Example: Player A played 90mins for 3 matches and 20min for 2 matches where he entered as a sub. So Games played = (90*3+20*2)/90 = 310/90 = 3.4 games played. The 310 value is from the Minutes data point in FM.
2 -
Just now, Mikal said:
@gam945 Thanks for this tool. I'm having some issues as below, do you know how to resolve?
When importing the shortlist on the excel sheet only a few names come through.
Naturally on the app there aren't any names that show up, but neither does other some of the other data such as value or wage. Also none of the visualisation selections show anything.
Yes, when you export from FM, make sure to scroll from the top to the bottom. Otherwise FM won't export all the rows. You'll see that if you open the HTML files they are also empty.
0 -
6 minutes ago, marculatur said:
I just fixed that problem for me: In Germany we use dots and commas not in the way they do it in GB.
To fix it, I had to change the following ingame options:
Preferences - Advanced - Formats
Positive: €1,00 (The Symbol of yout currency has to be before the value)
Decimal Symbol: Dot
Digit Grouping Symbol: Comma
With these settings, everything works fine for me now.
You nailed it, thanks!
0 -
I added two demo videos in Github under the demo_videos directory. There is one for the setup (only step I didn't do is double-click on installpackages.bat after saving it as I already have the packages), and one for data extract and app use.
@IWWROCKS I replicated the steps any user would do and didn't have any issues as you can see in the videos. Currently investigating to figure out the issue
0 -
1 minute ago, marculatur said:
Hi there,
thank you so much for this tool. I was looking for something like this a long time.
I just downloaded and tested your recent update. As you can see in the picture, everything runs fine so far. But i can`t see any data in the browser view. The correct club (RC Strasbourg) is displayed correctly, but every sheet is empty and no rows were found. The data in the excel file contains all the data from the views. Any suggestions?What i had to do to keep it running: I had to switch the ingame language to British English, as i come from germany and the translated categories were not able to import.
Adding this to the known issues, but yes, the data point names are in English, so the game should be in British English before exporting the data. The Excel tool will also not work if the data is not exported in British English.
Before exporting your data, change the game to British English. Then export and you'll be all set.
0 -
If you see a black window opening then immediately closing when you double-click on FMapp.bat, this is because you didn't change the paths! Please follow the instructions when you download the app to avoid this error!
Basically the FMapp.bat file contains the following:
<full path for Rscript.exe> <full path of app.R>
If you do not change the first path, Windows won't know which program (in this case Rscript.exe which comes with R when you install it) you are using.
If you do not change the second path, Windows won't know where is the script (in this case app.R in the main folder) you are trying to run from the Command Prompt.
0 -
2 hours ago, R0bby said:
Does it work with open office too, because i don´t have excel
Someone will have to test that...but as I mentioned in my first post, the Excel contains macros written in VBA. So maybe the excel will be compatible but someone will have to re-write the macros in the VBA equivalent for Open Office.
0 -
3 minutes ago, yanloup64 said:
Something went wrong. Double clicking on the batch file a black window pops up then closes immediately. I'm on Windows 7.
Which batch file? there are two. PM me as I just tested the batch files and there should be no errors.
Did you follow the instructions and edited the paths?
7 minutes ago, r70lla said:Brilliant idea, however, I'm having similar issues as Darj.
Fixed, please see my answer above.
0 -
2 hours ago, DarJ said:
I tried to import the squad data and I get this error. I don't know why because I tried with another save from the day of the job and it works fine so I don't know if the problem is because I'm at the end of the season on this particular save.
I went to look at the html file and that section is really missing so do I just edit the file on excel and add that column?
I was also unable to install the app and I don't know if this is related but after I changed the path for the file in the help folder and install it I see this this on command prompt
Thanks for raising this issue! the 4 columns Season <T,T+1> were the only one that referred explicitely to a year. This may change depending on the time in-game. Went on and removed those 4 columns, the Excel too is updated in Github. You could either just re-download the Excel tool or the entire folder if you don't feel comfortable.
As for the the install packages batch file, it has also been corrected. Credits to the first answer (https://stackoverflow.com/questions/26274553/install-r-packages-from-windows-cmd). Same as the Excel, you can re-download it from Github. However do not change the structure of the folder to avoid errors! The help folder shouldn't matter as nothing references to it, but to be sure do not change the structure
3 hours ago, IWWROCKS said:Really excited to dive into this!
Is there any way to dump all the players in the db out to pull into the reports?
Using the shortlist view, I can only add 327 players to a shortlist at a time. So if I wanted to run a report on a league, it can be quite difficult pulling all of the required players to the shortlist. Is it possible at all to run the shortlist export from the scouting screen instead so we could export all known players?
Edit: Ok so I worked out, I need to page down as FM can only add players to shortlist that it has seen. But this is still limited to 500 at a time.
Now, when i'm running the excel program (after exporting squad.html and shortlist.html and moving them to the folder) I am encountering an error "the column season 2021/22 of the table wasn't found" - run time error 1004
It has now been fixed, please check my answer just above.
0 -
App has now been released, ahead of schedule!
6 -
15 minutes ago, pilif25 said:
Is this going to work with FM21 only, or FM20 too? Also: how much are you planning on charging for it?
There are minor differences from FM20 but yeah after 2-3 modifications it should work.
From the top of my head, In FM20 the # of chances created was named "Ch C" while in FM21 it's "CCC", clear cut chances created. Also "xG" is not a stat in FM20. These modifications should be implemented to avoid errors when running the app. Nothing too complicated to do.
It will be free, and all the code will be open source, meaning that, quoting wiki, it "is released under a license in which the copyright holder grants users the rights to use, study, change, and distribute the software to anyone and for any purpose."
0 -
Just now, milestobudapest said:
Please make this open source. I would love to contribute. I have experience working with front ends and while I don’t have programming experience with R, I might be able to help with some of the backend logic too. Are you planning adding this to GitHub?
Of course this will be open source! I will post it on my GitHub and provide the link here for downloads.
I'm rushing to finish the app today (I'm 6 hours late from the UK time) so I can post it before sleeping lol
PM me if you're interested to contribute, help from someone with front end experience would really be appreciated
2 -
9 minutes ago, Peljam said:
Looking forward to this. Love using data for my saves so anything that speeds this up and makes it more accessible is great.
You mentioned injury data earlier on and using the squad views to extract. I’ve never been able to get much off screens that weren’t squad views. For a blog I’m doing I wanted to analyse data from the medical centre but print screen didn’t work. I had to do it manually. Are you able to get this data out as well?
yeah I can...
The tool will include an Excel extractor to pull the data from the html files extracted from FM
1 -
26 minutes ago, Hav3n_The_Hero said:
The definition of PPDA I could find states, "PPDA = Number of Passes made by Attacking Team (opponent) / Number of Defensive Actions , in the opponent's 60% of the pitch".
The last requirement may be the most difficult to get, but I have suggestion for that.
1. Number of passes made by attacking team. Which means to me number of passes completed not necessarily attempted.
2. Number of defensive actions include Possession-winning duels, Tackles, Interceptions, Fouls. I am not sure how possession winning duels is measured so I would ignore it, but tackles should include successful and failed tackles. Interceptions and fouls made by defensive team are pretty straight forward.
3. In the opponent's 60% of the pitch is very important and potentially difficult to measure.
3a. If you only measure all passes across the whole pitch it can make low block teams against possession based teams look much more effective at pressing then they actually are so it is critical this part can be measured.
3b. A potential solution to this is in the match analysis screen after a match is complete it has a section listed for the defensive half of the pitch and so it will only generate stats for that area of the pitch. I will share screenshots after to show what I mean.
3c. I do not know if it is easy to pull that data from football manager, but I do know it is generated I hope that helps.
I see what you mean... however I pull the data from the FM views, not from the after game match analysis. In fact I am 99% sure you can't pull data from it. Unfortunately I can't extract data relative to where in the pitch it has been recorded.
For the defensive actions, best I have in mind would be tackles attempted (not necessarily won) and interceptions. Fouls are likely to come from tackles so we'd count the same action twice... then again I don't know how SI records the match stats
0 -
12 hours ago, Hav3n_The_Hero said:
@gam945 So I have been thinking more about this tool as I am going to start an attribute masked masked run and will just rely on stats for most of my decisions. I do not know if you have it but a selection for Passes Per Defensive Action(PPDA) would be amazing. This would allow anyone to measure how effectively a player presses some players in a given game. Would be a wonderful scouting tool. Just a thought!
I am limited by FM's provided stats, but how would you suggest I do that? Passes completed vs (Tackles won + Interceptions) for example?
To give you an idea, I attached a dictionnary of all data points I extract from FM, you can take a look. I'd be more than pleased to add analysis wanted by the community!
Also, following is all the current visualizations I implemented:
For more clarifications:
Contribution refers to either Team goals scored or Team goals conceded
Rating consistency: games played vs average rating
Team results consistency: games played vs points per gameAll stats are per 90mins
2 -
11 minutes ago, Mike Trusky said:
My app is written entirely in C++. At the beginning I wanted to use python but I prefer C++, so I wrote everything in it. And yes, I've spent a lot of time on it, that's why it's not for free. (but really cheap :P).
In my app it depends on which tab you are currently using. Some only use in-game match stats, others have "variables" such as DNA, which can be set be each user.
And similar like you, I am working on exported html views from FM. (which I loaded into the app).
Yeah from the GUI I suspected C++/C/C#. Yours look more like a software, more professional while mine is more like a dashboard report.
Will have to teach myself C++ at some point, I was already planning on doing so. Re-writing my app in C++ would be a good starting project
How much lines/hours of code for your app out of curiosity?
0 -
1 hour ago, Mike Trusky said:
It looks really nice, I love that kind of stuff about analysis. If you'd like to check, sometime ago I released a tool that I wrote, including for analysis. It looks like this:
And at this link is an example of using the tool - Statistics Overview Example
Nice, seems to be close to my idea as well! Your app is built on what language? Mine is exclusively in R but I'm curious about yours. From what I see on your website the tool is not free so we probably won't have access to the source code (unlike mine) but I'd be keen to PM you to share ideas
I thought about letting the user define a "DNA attributes model" and use it to filter their shortlist as well but I preferred to go exclusively with in-game match stats.
0 -
Videos
Setup demo
App demo
V3 new features
UPDATE LOG: V3 (2021/02/21)
* "Home" menu added:
- Info boxes: number of players at the club & players in the shortlist
- Summary charts: bar chart of the goal contribution distribution, boxplots of the playing time and wages distribution
* "Progress" submenu added in the "Squad Analysis" menu: dates selection, performance and attributes progress between selected dates
* "Shortlist" submenu in the "Scouting" menu updated: now shows min and max asking prices and wage demands, presence of release clause and conditional release clauses instead of replicate table of the "Filters" submenus
* DT tables now used instead of Reactables: styling and formatting improved
* To use the V3 version you need to name your squad output files "squad_yyyymmdd", where "yyyymmdd" is the in-game date. This allows to compute the variations (progress) between given in-game dates. I updated the data folder in my Github to give you an example.
* It is possible thanks to the above feature to store multiple squad data files. Keep in mind that the more there is squad files in your data folder, the longer the app will take to load. You can find out how much time the different tasks (load data & format data for example) take time by paying attention to the command prompt that is launched when you double-click on FMapp.bat.
UPDATE LOG: V2 (2020/01/15)
* The app now directly reads the FM outputs to extract the data, thus meaning the Excel tool is obsolete. It is no longer included in the download.
* FFP Contribution and future time playing removed from the views (5 columns) as they are user-specific
* The app now dynamically install required packages if needed. First thing the code does is check if there are missing required packages and installs them. This means the installpackages.bat batch file is now obsolete. It is no longer included in the download.
* FMapp.bat now detects its current directory and uses it to launch the app. It is very important that you don't modify the file's structure! Instead of replacing 3 paths in the last version, you only need to replace 1, and it's a one-time process.
* DNA Model subsection was added in the Squad Analysis section. This enables you to select key attributes for your playing style and calculate a "DNA rating". Similarly to the Scouting section, the color code is used in the tables and the plots.
* Added status messages to show the user what's going on before the app is launched in their browser.
* Points in the plots have now a subtle black outline to make them more visible.
Bugs
[FIXED] "Current Season T,T+1" columns produce errors when extracting the html in the FM tool: removed those columns from the extract, as they will be variable depending on which season you are.[
FIXED]installpackages.bat not working: didn't wrote the R commands well, I fixed it with the help of stack overflow (First answer)[FIXED] Currency issues: replaced the "£" and "$" symbols in the code by their unicode characters.
Known Issues & Frequent errors
Before exporting data from FM, change the language to British English. All columns/statistics/data points are referred to their British English abbreviations. If anyone wants to do a translation, you're free to do it, it's open source.
FAQ (will be updated if I get the same questions over and over):
I noticed I'm getting warnings in the command prompt when my app is running, on certain plots... That's normal, the plots remove the data with missing values (say if a player has NA passes completed) so you'll have a warning that x observations are missing. You're actually missing missing data
There are no goalkeeper stats? Will be added in a future update, there is not a lot stats for goalkeepers apart from the saves and penalty conceded/faced...
In the Filters subsections, when I deselect all positions I get an error message instead of the table... Will be fixed in a future update but it has literally no impact. Just select at least a position and it'll go back to normal. Unless you really want to do an analysis on players with no positions?
THE APP DOES NOT WORK WITH MY SCREEN! I AM NOT A WEB DEVELOPPER!!! No seriously, I have 0 background in web dev, so I just went with fixed sizes in my code and made sure it was good for my 15inch screen. Just use it in the browser's max size (full screen), don't minimize it and you should be all good. I got some messages from people with experience in web dev, so hopefully I'll have helping hands on that matter. But again, full size in your browser and you should be all good.
Is it compatible with other OS than Windows? Well the batch files/command prompt is exclusive to Windows (I think so?). So you'll have to know a little bit of R, just enough run the packages install by yourself in R, then install RStudio, open the app.R file in RStudio and click on the Run App button. So that's less than basic knowledge of R. However I'm not sure about the compatibility of the Excel tool and the macros as they're written in VBA.
The app was able to run but it seems that the table and plots are empty... Before exporting the shortlist data from FM, make sure to scroll from top to bottom and/or Make sure that your in-game preferences are as following in FM before exporting:
On 09/01/2021 at 11:28, marculatur said:Preferences - Advanced - Formats
Positive: €1,00 (The Symbol of your currency has to be before the value)
Decimal Symbol: Dot
Digit Grouping Symbol: Comma
How can I contribute? Send me a PM
0 -
- Popular Post
- Popular Post
Hi fellow managers, I want to present you a new tool I'm working on that will enable us to use the data that FM provides us with at its full potential. Including attributes, diverse statistics, injuries data and many more, FM has more than 300 data points. Yet, I can bet any one of us doesn't even use the third of it to make decisions and manage our club.
Maybe it's due to the fact that FM doesn't offer an easy way of presenting all those data points. Take the match statistics for example, like the assists, passes completed, cross completion ratio etc. Yes, we could create a view with all those stats and compare them to decide which player on our shortlist will be the transfer target. But it is neither visually appealing to have a view full of numbers nor easy to make comparisons.
The app is now released! (2020/01/09)
The V2 Update is now released! (2020/01/15) Following is a quick guide to get started with the app.
Install R
You do not need to know anything about R, everything is programmed for you. However, you need to install the software. I will not go into too much details for this section as there are plenty of good guide, and as a quick Google search will yield the result. When you install R, remember in which directory it was installed as you will need it later.
Install R: https://cran.r-project.org/ > Download R for Windows > "install R for the first time" hyper link > "Download R 4.0.3 for Windows" hyperlink > follow the installation wizard instructions
** It is preferable to install R in a folder you have created in the C:\ directory (instead of installing it in "C:\Program Files") to avoid errors when running the batch file that launches the app
For example:
(1) Create the MyPrograms folder in C:\
(2) Install R in the following directory: C:\MyProgramsDo I need RStudio? No
Do I need to run something on R after installing it? No, patience is key, all what you need has been scripted by me
I don't know how to program in R! You don't need to know!Install the app
Go to my Github page: https://github.com/ybenadjal/FM-Data-Analytics > Repositories > FM-Data-Analytics
Download the complete folder then unzip it in your computer:
Unzip the zip folder you just downloaded (just copy the FM-Data-Analytics folder that is inside the .zip somewhere on your computer). Move the main folder(FM-Data-Analytics-main) wherever you want, but do not change the structure of the folder. Everything is now in the folder you just extracted:
Walk-through & set up the app: data
Contains the outputs you have exported from FM.
(1) Do not delete shortlist.html or squad.html if you don't replace it. Later running the app without one of the files may cause errors.
(2) Always name the outputs "shortlist" and "squad".
* V3 UPDATE: Name the squad outputs as follows: "squad_yyyymmdd", where "yyyymmdd" is the in-game date. You can now store multiple squad outputs
(3) When extracting the data from FM, be sure to scroll from top to bottom of the shortlist to get all the data when you use Ctrl+P to print and export.V2 UPDATE: The app now directly reads the FM outputs to extract the data, thus meaning the Excel tool is obsolete. It is no longer included in the download.
Walk-through & set up the app: demo_videos
Contains two video demos, one for the setup, and one for the data extract/use the app.
V2 UPDATE: Update to reflect the new workflow
Walk-through & set up the app: fmviews
Contains the views you have to import in your FM game, then export the FM screen as a Web Page/html file (Ctrl+P in FM).
completeview_shortlist_v4 is a view for the Shortlist in FM (Scouting > Shortlist).
completeview_squad_v4 is a view for the Squad in FM (Squad).** Do not change anything to avoid errors!
V2 UPDATE: FFP Contribution and future time playing removed (5 columns) as they are user-specific
Walk-through & set up the app: help
Contains a data dictionary. All the data points exported from FM are described here.
V2 UPDATE: The app now dynamically install required packages if needed. First thing the code does is check if there are missing required packages and installs them. This means the installpackages.bat batch file is now obsolete. It is no longer included in the download.
Walk-through & set up the app: main
Go back to the main file, there is the app: app.R, and another batch file to run the app: FMapp.bat
**IMPORTANT STEP (ONE-TIME ONLY)**
(1) Find your Rscript.exe file. It should be in the file you installed R > R-<version> > bin
(2) copy the path (in yellow)
(3) open FMapp.bat and replace your path in r_path1 (only replace the highlighted code in blue). That is the only adjustment you need to to!
(4) Save FMapp.bat and close it. You're set to use the app.The instructions are repeated in the setupdemo video in \demo_videos
****
V2 UPDATE: FMapp.bat now detects its current directory and uses it to launch the app. It is very important that you don't modify the file's structure! Instead of replacing 3 paths in the last version, you only need to replace 1, and it's a one-time process.V2 UPDATE: DNA Model subsection was added in the Squad Analysis section. This enables you to select key attributes for your playing style and calculate a "DNA rating". Similarly to the Scouting section, the color code is used in the tables and the plots.
V2 UPDATE: dynamic package installation.
V2 UPDATE: App reads html FM outputs instead of the Excel tool.
V2 UPDATE: Added status messages to show the user what's going on before the app is launched in their browser.
V2 UPDATE: Points in the plots have now a subtle black outline to make them more visible.
You're now all set! Launch the app by double-clicking on the FMapp batch file, the app will open in your browser!
The app will open in your browser. It is strongly recommended to set the zoom size as 90% and to maximize the browser's window
** When you close the app on your browser, also close the Command Prompt window that was opened by the batch file
TIPS:
(1) App interface
The app will launch in your browser. Set the zoom at 90% to get an optimal view. Use the 3 bars at the top to hide/show the sidebar, thus leaving more space for the content.
(2) Interacting with the plots
Some plots have trend lines, some do not. Hovering on the points will show you more info about the player. You can zoom in, navigate on the plot. Double click on the plot to get back to the default view. Quick demo:
Dots size: the more a player has games played (calculated by the total number of minutes ÷ 90), the bigger is the dot.
Dots color: depends on the scouting recommendation. Makes it easier to notice badly rated players that are in fact performing and vice-versa. Non scouted players are assigned -1, as well as squad players. V2 UPDATE: The DNA ratings have the same color code. If the user doesn't specify a DNA model then players are assigned a rating of -1.
(3) Shortlist builder
In the scouting section, you have a shortlist builder. This enables you to select some players you notice in each subsection (general/passing/shooting/etc). Then in the last subsection, Shortlist, you'll have the intersection of all the players you selected. Let's say you want a goalscorer AM. You noticed: Player A, Player B, Player C, Player D as good playmakers in the passing subsection and selected these players in the bottom shortlist builder. In the shooting subsection, you selected Player B, Player D, Player E and Player F. You'll see that in the Shortlist subsection, you'll have a table with Player B, Player D, thus your shortlist have been built.
** Leave all other Shortlist builders with all players selected if you don't use them. If you want to begin a new analysis and create a new Shortlist, make sure that all the shortlist builders have all players selected before beginning.
(4) Filters
Want to analyze your loaned players? Squad Analysis > Filters > Clubs: deselect your club (should be the first entry)
Want to analyze your players at the club? Squad Analysis > Filters > Clubs: deselect all then select your club (should be the first entry)
44 -
10 minutes ago, Junkhead said:
Right, I have today started my save, finally. After weeks of messing about trying to decide who to be, for some reason Como in Italian Serie C just came to me this morning completely out of the blue. I didn't question it, and have capitalised straight away and I'm up and running!
Decent squad, potential to grow and a stones throw from Milan and Genoa (who incidentally I would love to manage at some point...). Haven't had a decent Italian save for many versions.
Only done all the prelim stuff today, will be playing as slowly as normal, no doubt lol.
First friendly shows that the future is bright.... maybe....
Planning on doing my save with Padova in Serie C as well, nice to see another fellow manager in that league lol
Is the issue involving youth regens solved? Think I read in previous posts that there was not much regens generated in the future?
0 -
2 minutes ago, bielsadidnothingwrong said:
Doesn't even show up for mine anymore
it's a small detail, but it's really annoying...did you use custom logos or it doesn't show even with the unlicensed logos?
0 -
Hi guys, seems like the club logo is too much on the left on the unlicensed team's kits. Please see below:
Thanks.
1 -
Well maybe it's a return to normal...The 90s have so much world class players that were at least as good as Messi & Ronaldo...I think this era is an anomaly, the level of the payers have decreased a lot since late 2000s. So, nice to see that the level increases in FM.
0
FM Data Analytics: a data visualization tool based on R & Shiny - V3 RELEASED!
in Football Manager General Discussion
Posted
Nothing, it just installs the packages then it closes automatically. You should notice success install messages in the lines of codes tho. If there's an error with the install it will pause, so if it closes after running it should be all good.