It's been awhile since we mentioned our work on the new mod manager. For a quick recap of "recent" events, back in August 2016 we brought on Tannin (of Mod Organizer fame) to head up work on a completely new mod manager for Nexus Mods. He's been working on it along with the two original NMM programmers ever since.

Tannin's remit is simple on the surface, we want a mod manager for Nexus Mods that can handle modding for as many games as possible and that can be both simple for newcomers to use while containing (or supporting) all the advanced functionality users have come to expect from mod managers for games like Skyrim and Fallout. Simple on the surface, not so simple when you get down to it!

Tannin has been given a very wide berth to decide the how and the when of things. My thinking is that Tannin was able to make a great piece of software in his spare time while also working a full-time job, so I really don't need to tell him how to do things -- he knows how to do things. And I'm certainly nowhere near as qualified to make the choices he's making for the good of the mod manager. So, Tannin is most definitely the project lead and I just comment on (and argue with Tannin about) UI related things. Something I'm sure he can attest to!

Since details about the new mod manager are thin at the moment (we're simply being tight-lipped as we're not ready to talk about too much yet -- we'd rather work on it than be constantly answering questions or doing PR!), I thought we could do a little Q&A about the things we can now talk about as we head towards a closed alpha of the software.

What I can tell you now is that we have agreed (this week!) on a name for the new mod manager. It's taken us a long time to come up with one mainly because I was looking for some particular criteria in the name. I wanted a single word name that was interesting, marginally relevant and "brandable". While this might sound odd, I also didn't want a name that was specific to mods. Who knows what this software will become in 2, 5 or even 10 years time. For example, I'd like at some point to provide a cloud save game storage service to members of Nexus Mods so they can easily backup and restore their saved games. At such a point, we've gone past a mod manager and the name is no longer relevant. I'd rather just have a more generic software name now, than have to "rebrand" in the future.

I vetoed things like Nexus Client (though it was a fallback!), Nexus Mod Manager 2 and Nexus Mod Organizer because they're either boring, cumbersome or both. Instead, we took up a suggestion from the last news post we made about NMM from users in our community. I asked around a little and most people polled liked it, so...done!

I can confirm that the new mod manager at Nexus Mods will be called Vortex.

We'll begin working on a logo soon.

With that out the way, I've already done the introduction to Tannin, so without further ado let's get on with the questions and answers…

Robin: So first things first, Tannin, what would you say are the key differences between your goals and your ethos working on Mod Organizer compared to your goals and your ethos working on Vortex?

Tannin: When I started Mod Organizer, the virtual file system software that made it special was already mostly done. I had created the vfs for a separate purpose and when I played around with mods for Oblivion it occurred to me that this tech could be applied to that problem.

I never really expected a lot of success and for the first year or so I didn’t have any. I was always in it for the technical challenge and my attitude was, "This is what I have, if others like it I'm happy, otherwise they have alternatives". It was never my goal to create the best possible mod manager for everyone, I was happy in a niche.

Now with Vortex that has changed of course. We're making the official mod manager for Nexus Mods and it has to be accessible and trouble-free for everyone.


Robin: I think we both know the biggest questions we've received around Vortex have been in regards to virtualisation and how Vortex will handle and store files on people's hard-drives. Is Vortex going to use virtualisation?

Tannin: Yes it does.

I know people have - often very strong - opinions on the topic so I ask that you please read my reasons before you go to the comments and vent.

In the initial release of Vortex, virtualisation will be implemented using links (symbolic or hard links), similar to NMM v0.6. We've left the door open so we can implement different approaches (i.e. the usvfs library from Mod Organizer) but at this point I don't think there will be a "no virtualisation" option.


Robin: Can you explain why Vortex is going to be using virtualisation and why it's preferred to using the old method of simply extracting mod files into the game directory?

Tannin: I'll try to be brief!

Managing mods without virtualisation is slower, takes more space on your hard-drive and has a lot of hidden complexity that makes it more error prone as well as more work to implement (this translates to "fewer features per time" for those of you who don't care about a programmer's pains) and harder to understand.

It's best to work with an example to demonstrate. Imagine:

• You install mod X that contains "spider.dds" and "bear.dds"
• And you install mod Y that contains "spider.dds" and "dragon.dds"
  

You expect "spider.dds", "bear.dds" and "dragon.dds" to be in your game directory and "spider.dds" can be either the one from mod X or the one from mod Y, depending on the order of and choices made during installation. Let's assume it's from mod Y.

You then decide you don't like it and remove mod Y.

You now expect "spider.dds" and "bear.dds" to be in your game directory and "spider.dds" should now be the one from mod X.

To make this happen, a mod manager that doesn't use virtualisation needs to keep a manifest of where files come from and which files have been overwritten so that it can know there is another "spider.dds" in mod X and how to restore it.

Once this mod manager knows "spider.dds" exists in mod X it has to open the archive from which you originally installed it and extract the file again. This manifest is hidden in some database or xml file, probably invisible or unreadable to you. Most users will not know what information it contains or how it's used. You may not be aware you need to back it up together with the game directory, but the content of the manifest needs to be in sync with the actual installed mods.

This is a hidden complexity to the mod manager that you may not care about until it breaks.

If you restore a backup that contains an older manifest but not the mods or the other way around, your mod manager will no longer be able to correctly disable/remove/change the order of mods. And this is unrecoverable.

The same is true if you manually install a mod or remove a file from the game directory that the manager was tracking in its manifest. If you do this you're invalidating the manifest and forcing the manager to guess what has happened. It may guess correctly 9 out of 10 times, the other time you will get an error.

The manifest also needs to be migrated. Every time we change the manifest format we need to develop a migration path from the old format to the new format. If that migration code has a bug or can't deal with an already broken manifest it's potentially breaking the mod installation for thousands of users. Errors in the manifest can remain lingering until some day the manifest migration fails.

It also means you have to keep the archives for all installed mods around and if you delete one file for that mod, it can't be restored.

Similarly, extracting from an archive can be very slow, especially if it was heavily compressed to save space and bandwidth.

Finally, if you have modified "spider.dds" from mod X prior to installed mod Y those modifications are going to be lost. Whoops!

On the other hand, mod managers that use virtualisation, like Vortex, don't have to track individual files. They just need to know what mods you have (which is obvious since they are a bunch of directories on your disk) and how to prioritise them.

The "deployment" of files to the game directory is then simple, using the list of enabled mods and their priority at that point in time to reliably determine which files should be there. There is no need to know what happened in the past. When installing mod X and mod Y, it deploys the files from both mods to the game directory, the "spider.dds" from Y overloads that of X. Everything is as expected.

When mod Y is removed, it also removes all the files that mod Y deployed to the game directory (which it can detect without the need of a manifest because the links themselves tell us where each file came from) and then deploys a new. This time mod X gets to deploy "spider.dds".

Everything is as expected - without a manifest, without extracting files and as an added bonus, if the file was modified in mod X, those changes are still going to be there.

Therefore (and TL; DR) virtualisation is actually simpler, quicker and also saves space as you don't need the mod archives after installation (you can keep them if you want but it's optional).


Robin: That wasn't very brief. Why not make virtualisation an option that users can pick from?

Tannin: We may, at some point. However, to be honest, virtualisation is objectively superior and to implement a "no virtualisation" installation method is a lot of work both to develop and to maintain (for the reasons mentioned above). Without good reasons, it's just not a good use of our time when there are so many other useful features we could be doing right now.


Robin: You've already said that the virtualisation system Vortex uses will be more like the one NMM uses than the one Mod Organizer uses. You and I both know there are a lot of MO users who swear by their clean install folders and are likely to be upset by this. Can you explain your reasoning behind choosing the NMM way over the MO way?

Tannin: Well first of all, providing the "MO" vfs as an option at a later time is not off the table for Vortex. But there are a few reasons why having the MO vfs as the default option for Vortex wouldn't be a good idea.

Firstly, the way the vfs is implemented is more error prone. In particular, it's more likely to encounter incompatibilities with different applications. This was acceptable (to me) with MO where only a handful of games were supported with a few tools for each, but even then we had frequent problems with tools not running/producing odd errors that could take weeks to fix.

With Vortex, we want to support as many games as possible and fixing the vfs for each game and each tool for dozens of games would be a maintenance nightmare.

Secondly, the heuristics of virus scanners frequently (and incorrectly) tagged MO as a virus because the technology used to implement the vfs is similar to what some malware does. Again, this was ok when it affected a few percent of a few thousand MO users who tended to be more tech-savvy than the average user. However, Vortex aims to be accessible to all users and we really don't want bug reports about AV errors from 5% of 6 Million users…

Thirdly, I'd like to point out that when I initially started developing MO, the vfs technology (the 32-bit variant) was already mostly done as I had created it as part of my diploma thesis and merely rewrote that. So, really, I had a solution and was looking for a problem to solve with it.

MO was my experiment to see if this technology could be used for that problem. My goal with MO initially was not to write the most user-friendly Mod Manager I could. If I'd had the time I would have added a link-based option to MO2 as well...

While a clean game folder solution isn't possible right now with Vortex, there will be a "purge" feature which will remove all links installed by Vortex reliably and quickly so you shouldn't ever be more than a button-click away from a clean game directory.


Robin: Is there a performance cost to using virtualisation?

Tannin: For hard-links: no, none, zero.

For symbolic links: practically none. I doubt you could measure it and I can guarantee you won't be able to notice it.


Robin: Some users have reported and complained about their custom mod backups being twice the size they should be due to virtualisation. What's the solution to this?

Tannin: Get better backup software. Seriously, there is no excuse for a backup solution to not handle links properly.

But as an alternative, let me again refer to the "purge"-option that un-deploys all mods. You can do that before creating the backup, then just have Vortex re-deploy afterwards. This is a safe operation and will probably take less than a minute.


Robin: Moving forward, what are your release plans for Vortex? Will there be an alpha? What time scales are we talking here?

Tannin: Giving concrete dates is always difficult because one almost always underestimates the amount of work required to polish stuff towards the end.

My current plan is to have an early alpha build in the hands of a limited group of test users within a month, maybe 6 weeks.

Depending on their feedback we should expect somewhere between 1-3 months to fix bugs after which I think we can release a public alpha.