Note: there is a bug in v1.3 and v1.4 that omits too many lines in some situations. If you feel that some lines are missing, use v1.2 - it may include too many lines but at least you should have everything. I shall be reviewing the scripts and will post an update when ready.
While using it, I noticed that it could output invalid CSV lines if there are double quotes in the dialog text. You can fix this by replacing lines any of these lines:'"' + GetElementEditValues(response, 'NAM1') + '"';with '"' + StringReplace(GetElementEditValues(response, 'NAM1'), '"', '''', [rfReplaceAll, rfIgnoreCase]) + '"';
This just does a replace for any doublequote characters and turns them into single quotes.
Thanks for the tip! However, simply replacing two double quote character with one will not solve the problem in all cases. Depending on where the double quotes were in the dialogue text, sometimes you need to replace them with one character, sometimes they need removing altogether, and other times you need to manually append a double quote character to the end of the text field. It would be a better approach to remove all double quotes from the dialogue text before extraction, or better yet to replace them with a single quote which is handled correctly. This is why I say that the CSV needs post-processing. Text fields also often contain other invalid strings such as instructions for the speaker or substitute variables, all of that needs cleaning before importing into xVAsynth.
The new lines just look for any double quotes in the dialog text and replaces it with a single quote. Should work for any number of double quotes. With Pascal, you put in two single quotes to escape it, so it knows you are inserting a literal single quote character.
Here is what the output looks like:
current functionality "She will ask you "Have you seen the Twin Lamps?" Respond with "They light the way to freedom." Then she will know you are a friend." proposed functionality "She will ask you 'Have you seen the Twin Lamps?' Respond with 'They light the way to freedom.' Then she will know you are a friend." The problem I am running into with the current setup is I can't load the CSV in a script to perform the post processing automatically since it has a possibility to output invalid CSV. I have been using this for a while and thought I would share if it helps.
Oh I see! I am sorry, I misunderstood your original post. I thought you were replacing two double quotes with one double quote and that wouldn't work. But replacing that with a single quote would work indeed! I shall update the script, thank you!
The problem with variables such as &stuff; still remains though, as well as text such as [remain silent]. There is a lot of that in Morroblivion.
Yeah, the output isn't perfect, but at least it will be a valid CSV file. I used to have to export dialogue, look at it with a CSV linter, correct any mistakes, then feed the new file into my script.
This way I can just feed the file into my script and clean up the dialogue with that.
As for the &variables in dialogue, I normally just ignore those haha.
This continues to be extremely useful with the addition of the scripts to filter for missing mp3/lip files. I see that the scripts don't recognise the GetIsRace condition checks on the speaker, so where e.g. a line is spoken only by any Argonian, the missing mp3/lip files scripts will state that every other race is missing their files.
For the dialogue lines I have, the condition checks are first for GetIsRace, followed by GetIsSex - the gender condition is recognised in your scripts. EDIT Actually, it seems that it can cope with either the race condition, or the gender condition, but not both combined. The results list the correct gender for every wrong race, and the wrong gender for the correct race.
The scripts are currently not able to process multiple conditions at once. They look at race first, and if that is not specified, they look at sex, but not at both. These scripts are based on the older one that you find among the standard scripts, hence the limitation. I would need to rewrite them completely to get that fixed. Might take a while.
If I understand this properly, the only issue is that it will generate redundant voice lines, or is there any other issue that may still leave silent lines?
Additionally, would it be easy to adapt this script for Skyrim SE? I suppose that the main change would be updating the voice models references.
Hi, yes, the issue is (I think) with the way the script is, it generates too many voice lines. I haven't tested it extensively with both conditions though. But I believe it generates too many rather than not enough.
Adapting for Skyrim takes more than model names - Skyrim dialogues have a somewhat different structure. I do not work with Skyrim, so couldn't do it, but there are existing Skyrim scripts and they don't work for Oblivion.
Thanks, then I will likely see to do it in Mutagen Synthesis since it should be more future proof. I already have a project to generate the voice lines for the Dragonborn Voice Over mod, and filling the silent NPC lines would be the last thing remaining.
For now, I am finishing generating around 50000 dialog lines after using this over the MOFAM modding guide and trimming down repeated voice lines.
Just for future reference. To make it work I had to apply the following main changes to the csv: * Replace .mp3 for .wav since xVASynth crashes when using ffmpeg if the source is an mp3 * Remove the content between '[' ']' since that tends to generate random errors and it is used to explain the context not as a voice. * Replace the double quotes inside of the sentences for single quotes. * Remove duplicate lines. * Remove lines with empty text. * Sort lines alphabetically.
In addition, instead of using CSE for the lip files I am using the plugin available for xVASynth in Skyrim Nexus. Then Yakitori Audio Converter to convert the .wav into .mp3
I had to do the same changes for Oblivion, except that I have no issues generating mp3 with xVAsynth. My knowledge of Pascal and xEdit API is insufficient to program this into an xEdit script. The CSV does need post-processing, but any decent spreadsheet software will be able to do it.
Did you get around to fixing the conditions in the scripts, LenaWolfBravil? It doesn't just add extra false entries, it also leaves valid entries out. For example, a dialogue line where every playable race can say it, but Breton NPC Mike cannot, will only export one line, for "Breton\M", completely missing the other 10 vanilla entries which should also have been exported.
Sorry, I've had no time to look at it. It requires a complete rewrite in order to improve condition handling, and I have to do a lot of learning before I can do it.
The incorrect handling of negation however is an error. I shall look at that first, although not immediately.
Well, I have written a Mutagen/Synthesis dialogue export module. This module generates an optimized .csv file with all the missing mp3 dialog lines. It takes into account the NPC, Faction, Class, Race and Sex conditions of the subject to reduce the number of required lines to generate and generates potential voice styles that could be used if defined manually in xVA Synth. In addition, I have added some ducktape techniques to resolve most of the errors that I have found so far (like the expression "Am I ...", the crash when using the word "Amazing", the weird spelling of "ok" and the csv error with double quotes)
I have also added support for Children Of Cyrodiil using the voice models from Skyrim (expect for Argonians and Khajiit which will use the adult models).
Lets see if I manage to upload it to GitHub or wherever it need to be able to be shared. Meanwhile if anyone wants to give it a try locally and give me some feedback, just DM me and I will share the Project Solution to be manually imported in Synthesis.
Checking Faction may not be desirable. Factions are dynamic, NPCs go in and out during gameplay. You may be removing too many lines - the NPC may not be member of a faction to start with, but would join it later and not find the lines to say.
And just to be clear: your script is for Skyrim, right? Not useable for Oblivion?
My script is for oblivion. In regards to factions, I am checking the race and sex of all NPC referencing the faction. If that issue really happen, that is a minor fix. The same logic could be applied to be made compatible with Skyrim but the mutagen API is not as developed in Oblivion and I have had to ducktape a bit.
Out of 51709 lines generated by the script in my mod list, only 1419 has been trimmed down by the faction condition. So that would have a minor impact if removed.
I am also thinking in optimizing the script a bit more by creating a csv which each line appearing only once and generating a bat file to copy paste that file in each of the other locations that the same sentence is used by the same voice model.
Mutagen/Synthesis is a framework developed in c# to apply patches dynamically like with xEdit script. Is well known in the Skyrim community. For Oblivion I have written some proof of concepts for tasks like, extended support for "Balanced NPC Level Cap - Logical Levels for All NPCs", "Complete Clutter and Crop Ownership", "Weapon Reach And Speed Overhaul" and a dynamic OCRAFT Integration System which creates more types of ingots, assigns new pelts for animals, creates new variants of jewerly using the different qualities of pearls/diamonds..., also creates variants of fine and durable for all standard weapons and replaces some rocks (like MOO) with veins.
Exactly, nothing to do. In the end, if you know some programming, you could just end up using both, xEdit to look at the registers and conflicts and Mutagen to write quick and dirty patchers which will have the output merged in a single .esp. I suppose that it would even be possible to replace zEdit with some effort as well.
I get an error when I try to run this script: Error in unit 'OblivionExportDialogues' on line 27 : Type of expression must be boolean TesEdit 3.2.1. Any ideas how to fix it?
Thank you, this is very handy. Just a couple weeks ago I was looking at the results of the existing export dialogue xEdit script and wishing it separately listed the same line once for race/gender expected to speak it, now with your script I have that data.
31 comments
Thanks to xenogearslucas for reporting it.
While using it, I noticed that it could output invalid CSV lines if there are double quotes in the dialog text. You can fix this by replacing lines any of these lines:
'"' + GetElementEditValues(response, 'NAM1') + '"';with'"' + StringReplace(GetElementEditValues(response, 'NAM1'), '"', '''', [rfReplaceAll, rfIgnoreCase]) + '"';
This just does a replace for any doublequote characters and turns them into single quotes.
Thanks!
However, simply replacing two double quote character with one will not solve the problem in all cases. Depending on where the double quotes were in the dialogue text, sometimes you need to replace them with one character, sometimes they need removing altogether, and other times you need to manually append a double quote character to the end of the text field. It would be a better approach to remove all double quotes from the dialogue text before extraction, or better yet to replace them with a single quote which is handled correctly. This is why I say that the CSV needs post-processing. Text fields also often contain other invalid strings such as instructions for the speaker or substitute variables, all of that needs cleaning before importing into xVAsynth.
Here is what the output looks like:
current functionality
"She will ask you "Have you seen the Twin Lamps?" Respond with "They light the way to freedom." Then she will know you are a friend."proposed functionality
"She will ask you 'Have you seen the Twin Lamps?' Respond with 'They light the way to freedom.' Then she will know you are a friend."The problem I am running into with the current setup is I can't load the CSV in a script to perform the post processing automatically since it has a possibility to output invalid CSV. I have been using this for a while and thought I would share if it helps.
The problem with variables such as &stuff; still remains though, as well as text such as [remain silent]. There is a lot of that in Morroblivion.
This way I can just feed the file into my script and clean up the dialogue with that.
As for the &variables in dialogue, I normally just ignore those haha.
Thanks again!
For the dialogue lines I have, the condition checks are first for GetIsRace, followed by GetIsSex - the gender condition is recognised in your scripts.
EDIT
Actually, it seems that it can cope with either the race condition, or the gender condition, but not both combined. The results list the correct gender for every wrong race, and the wrong gender for the correct race.
Additionally, would it be easy to adapt this script for Skyrim SE? I suppose that the main change would be updating the voice models references.
Adapting for Skyrim takes more than model names - Skyrim dialogues have a somewhat different structure. I do not work with Skyrim, so couldn't do it, but there are existing Skyrim scripts and they don't work for Oblivion.
For now, I am finishing generating around 50000 dialog lines after using this over the MOFAM modding guide and trimming down repeated voice lines.
Just for future reference. To make it work I had to apply the following main changes to the csv:
* Replace .mp3 for .wav since xVASynth crashes when using ffmpeg if the source is an mp3
* Remove the content between '[' ']' since that tends to generate random errors and it is used to explain the context not as a voice.
* Replace the double quotes inside of the sentences for single quotes.
* Remove duplicate lines.
* Remove lines with empty text.
* Sort lines alphabetically.
In addition, instead of using CSE for the lip files I am using the plugin available for xVASynth in Skyrim Nexus. Then Yakitori Audio Converter to convert the .wav into .mp3
The incorrect handling of negation however is an error. I shall look at that first, although not immediately.
I have also added support for Children Of Cyrodiil using the voice models from Skyrim (expect for Argonians and Khajiit which will use the adult models).
Lets see if I manage to upload it to GitHub or wherever it need to be able to be shared. Meanwhile if anyone wants to give it a try locally and give me some feedback, just DM me and I will share the Project Solution to be manually imported in Synthesis.
And just to be clear: your script is for Skyrim, right? Not useable for Oblivion?
Out of 51709 lines generated by the script in my mod list, only 1419 has been trimmed down by the faction condition. So that would have a minor impact if removed.
I am also thinking in optimizing the script a bit more by creating a csv which each line appearing only once and generating a bat file to copy paste that file in each of the other locations that the same sentence is used by the same voice model.
Also: what is Mutagen?
Error in unit 'OblivionExportDialogues' on line 27 : Type of expression must be boolean
TesEdit 3.2.1. Any ideas how to fix it?
I tried it in version 4.0.4 and it works. So it's a problem with Tes4Edit.