Working in progress...

1. Basic Optimizations
Close or minimize unnecessary programs.
Make sure they are exited/disabled/minimized before you play SkyrimSE:

1. Web Browser
2. Downloaders
(Especially P2P like BitTorrent)
3. Antivirus Programs
(Or turn Gaming-Mode on if available)
5. Any other CPU/RAM/GPU heavy programs
(Visual Studio, Photoshop, Video Players...)
6. Any other games

Fullscreen
This may improve your FPS significantly.
You should use Fullscreen when you have FPS issue.
Although It can make Alt+Tab a bit annoying


You can find more technical details on:
Desktop Window Manager - Win32 apps | Microsoft Docs (microsoft.com)
Desktop Window Manager - Wikipedia (wikipedia.org)
Demystifying Fullscreen Optimizations | DirectX Developer Blog (microsoft.com)

Set Priority
Skyrim Priority SE at Skyrim Special Edition Nexus - Mods and Community (nexusmods.com)
Skyrim Priority improves Skyrim CPU utilization by increasing process priority of Skyrim
Make Skyrim always take most of CPU usage
Increase FPS and Prevent suddenly lag caused by other processes

You can find more technical details on:
Scheduling Priorities - Win32 apps | Microsoft Docs (microsoft.com)

2. GPU Optimizations
GPU is not directly controlled by the operating system.
To optimize GPU you will have to go GPU Driver Control Panel.

NVIDIA Control Panel
This may improve your FPS EXTRAORDINARILY.
How To Reduce Lag - A Guide To Better System Latency (nvidia.com)
If you can't find these options in your Control Panel.
You may need to update your NVIDIA Driver. (Game Ready Drive)
Download Drivers | NVIDIA (nvidia.com)

AMD Radeon Software
I'm not AMD Graphics Card User. So I can't help you with this.
But there's some link to articles may help you.
How to Configure Radeon™ Chill, Radeon Anti-Lag, and Radeon Boost | AMD (amd.com)
Configure AMD Radeon™ Settings for Ultimate Gaming Experience | AMD (amd.com)
AMD users, maybe helpful :: The Elder Scrolls V: Skyrim Special Edition General Discussions (steamcommunity.com)
If you have any good solution you can comment here

SSE Display Tweaks
SSE Display Tweaks at Skyrim Special Edition Nexus - Mods and Community (nexusmods.com)
SSE Display Tweaks Mod improves GPU performance in Borderless Mode by using DirectX Flip-Model
Also included many thing like bug-fixes and performance optimized physics fix...

You can find more technical details on:
For best performance, use DXGI flip model | DirectX Developer Blog (microsoft.com)

3. Memory Optimizations
Increase Page Size
[ This won't improve your fps directly. But it may help to improve RAM cache size]
[ Improve Program startup time after first startup ]

Press "Win+R" and type "sysdm.cpl" then press "Enter" to open "System Properties"
Go "Advanced" tab. Click "Settings..." button in "Performance" box.

Go "Advanced" tab. Click "Change..." in "Virtual memory" box.

It's recommend to put paging file to SSD Volume
Just don't make page file too large that even larger than your volume available space.

4. Disk Optimizations
Defragment
[ You can ignore this step if you are using SSD (Solid state drive) ]

Press "Win+R" and type "dfrgui.exe" then press "Enter" to open "Optimize Drives"
Optimize all items that Media-type Is Not "Solid state drive"
(In this image, they are optimized for example purposes)
(You shouldn't do that)

This may slightly improve your Loading Time and Game Saving Time for HDD.

Ensure "Write-caching" is Enabled 



5. Other Optimizations
Disable Fullscreen Optimizations
[ Only Windows 10 needs this ]
[ Only for people playing game with Fullscreen mode ]



6. Controversial Optimizations
<Game Mode> in Windows 10
Although it named "Game Mode".
But many players reports it will actually reduce performance.

Disable Services
Modern Windows is smart enough to manage services itself.
Most of services will only cost CPU cycles when they are needed.
Do this only you clearly know what are you doing.
Otherwise some weird troubles may happen.
(e.g. Disable "Spooler" will cause "OneNote UWP" unable to update)