The success of generalization for segmentation depends not only on the network configuration, but also on the augmentation and pretrain on non mask target.

try my new project Deep Roto https://iperov.github.io/DeepXTools/

looks similar to "Cold Diffusion"

exception:

Windows not yet supported for torch.compile

torch.compile does not work in windows :(

so, technically this is binary optimizer that updates the weight to either -1 or +1 multiplied by lr. Should be tested with "Learning Rate Dropout", i.e. 30% chance to update with -1/+1, otherwise no update.

112x112 resolution. Completely useless in 2k23

use trainable normalization

self._in_beta = nn.parameter.Parameter( torch.Tensor(in_ch,), requires_grad=True)
self._in_gamma = nn.parameter.Parameter( torch.Tensor(in_ch,), requires_grad=True)
...
self._out_gamma = nn.parameter.Parameter( torch.Tensor(out_ch,), requires_grad=True)
self._out_beta = nn.parameter.Parameter( torch.Tensor(out_ch,), requires_grad=True)

...

x = x + self._in_beta[None,:,None,None]
x = x * self._in_gamma[None,:,None,None]
...
x = x * self._out_gamma[None,:,None,None]
x = x + self._out_beta[None,:,None,None]        

ML is not only just the backend. Technically you can code and run ml programs on OpenCL or OpenGL, but speed will be at least x2-x4 worse than specialized backend like cuda / rocm.

It's all about tuning programs (such as matmul) for each GPU model to achieve maximum performance. CUDA/Rocm already contains tuned programs.

Because each project has its own configuration.

If you make a framework out of this, you get a horror like Bazel. And you will spend time learning how to work with Bazel. But what for? Building a project is a simple operation: create folders, some files, download, unzip, call Popen with a certain env, clean up the __ pycache __ folder at the end, and archive it - that's it! The project is ready.

All you need to do is spend 20 minutes and figure out how WindowsBuilder.py works and adapt it to your project.

WindowsBuilder.py is standalone and requires only python 3.

what do you mean

look at my project https://github.com/iperov/DeepFaceLive

I made a builder to create an all-in-one standalone folder for Windows that contains everything to run a python application containing cuda runtime. Release folder also contains portable VSCode that has already configured project to modify only folder's code. No conda, no docker and other redundant shit.

Builder located here https://github.com/iperov/DeepFaceLive/blob/master/build/windows/WindowsBuilder.py and can be expanded to suit your needs.

so, with torch.compile people can keep writing graph-unfriendly code with random dynamic shapes and direct python code over tensors ?