Running bark ai with pytorch 2 on a rx580 2048SP 8gb

Bark is an AI text to speech generator

WARN: You will NOT be able to run the full sized model with this card, you need 12gb of VRAM for that. You can however use the small model

Following the guide here you can get the last supported version of rocm installed

sudo echo ROC_ENABLE_PRE_VEGA=1 >> /etc/environment
sudo echo HSA_OVERRIDE_GFX_VERSION=8.0.3 >> /etc/environment
# reboot

wget https://repo.radeon.com/amdgpu-install/22.40.3/ubuntu/focal/amdgpu-install_5.4.50403-1_all.deb
sudo apt install ./amdgpu-install_5.4.50403-1_all.deb
sudo amdgpu-install -y --usecase=rocm,hiplibsdk,mlsdk

sudo usermod -aG video $LOGNAME
sudo usermod -aG render $LOGNAME

WARN: Do NOT use default options for amdgpu-install (run without any flags)! This will install a non-working driver and your card will bootloop until the software is uninstalled!

Then you install to install NCCL. If you have a nVidia dev account you can download the deb, if not you'll need to compile it.

git clone https://github.com/NVIDIA/nccl.git
cd nccl/
make -j src.build

WARN: It took 16gb of ram and 32gb of swap for me to compile it. If you don't have enough swap run

sudo fallocate -l 32G ./swapfile
sudo chmod 600 ./swapfile
sudo mkswap ./swapfile 
sudo swapon ./swapfile

You will also need to install nvidia-cuda-dev, but this is in Ubuntu repos!

sudo apt install nvidia-cuda-dev

From there run

git clone https://github.com/pytorch/pytorch.git -b v2.0.1
cd pytorch
export PATH=/opt/rocm/bin:$PATH ROCM_PATH=/opt/rocm HIP_PATH=/opt/rocm/hip
export PYTORCH_ROCM_ARCH=gfx803
export PYTORCH_BUILD_VERSION=2.0.1 PYTORCH_BUILD_NUMBER=2
python3 tools/amd_build/build_amd.py
USE_ROCM=1 USE_NINJA=1 python3 setup.py bdist_wheel

After compilation is finished the .whl file should be under /dist/

I strongly recommend NOT installing this system wide. Make a virtual env for the project you're going to use it in and install it there.

python -m venv venv --system-site-packages

source venv/bin/activate
pip install /path/to/pytorch-2.whl

Now you're ready to install torch audio

export BUILD_VERSION=2.0.2
git clone https://github.com/pytorch/audio.git
cd audio
mkdir build
cd build
MAKE_CXX_COMPILER=/usr/bin/hipcc cmake -DBUILD_BENCHMARK=ON -DCMAKE_PREFIX_PATH=/home/$USER/code/bark/venv/lib/python3.10/site-packages/torch  ../.
FORCE_CUDA=1 ROCM_HOME=/opt/rocm/ python3 setup.py bdist_wheel

Make sure you build version 2.0.2! It's the only version that works with pytorch 2.0.1!

From start to finish counting download (pytorch is fucking massive!) it took me a little over 8 hours to get everything working.

Cuda init error:

To bypass the cuda init check (which our GPU will crash because cuda assumes we have a nVidia named card not gfx803) open the file venv/lib/python3.10/site-packages/torch/cuda/http://__init__.py, search for def _check_capability(): and add return under it. It should end up looking like

def _check_capability():
    incorrect_binary_warn = """
    Found GPU%d %s which requires CUDA_VERSION >= %d to
     work properly, but your PyTorch was compiled
     with CUDA_VERSION %d. Please install the correct PyTorch binary
     using instructions from https://pytorch.org
    """

    old_gpu_warn = """
    Found GPU%d %s which is of cuda capability %d.%d.
    PyTorch no longer supports this GPU because it is too old.
    The minimum cuda capability supported by this library is %d.%d.
    """

    if torch.version.cuda is not None:  # on ROCm we don't want this check
        return
        CUDA_VERSION = torch._C._cuda_getCompiledVersion()
        for d in range(device_count()):
            capability = get_device_capability(d)
            major = capability[0]
            minor = capability[1]
            name = get_device_name(d)
            current_arch = major * 10 + minor
            min_arch = min((int(arch.split("_")[1]) for arch in torch.cuda.get_arch_list()), default=35)
            if current_arch < min_arch:
                warnings.warn(old_gpu_warn % (d, name, major, minor, min_arch // 10, min_arch % 10))

You are now ready to test your bark install

from bark import SAMPLE_RATE, generate_audio, preload_models
from scipy.io.wavfile import write as write_wav
from IPython.display import Audio
import os
os.environ["SUNO_OFFLOAD_CPU"] = "True"
os.environ["SUNO_USE_SMALL_MODELS"] = "True"

# download and load all models
print("Preloading models")
preload_models()
print("Loaded")

# generate audio from text
text_prompt = """
     Hello, my name is Suno. And, uh — and I like pizza. [laughs] 
     But I also have other interests such as playing tic tac toe.
"""
print("Generating text")
audio_array = generate_audio(text_prompt)
print("Done")

# save audio to disk
write_wav("bark_generation.wav", SAMPLE_RATE, audio_array)
  
# play text in notebook
Audio(audio_array, rate=SAMPLE_RATE)

Save the above to a file named test.py and run it.

YOU SHOULD NOT SEE ANY WARNINGS ABOUT AN UNUSED GPU. IF YOU DO REINSTALL TORCH pip install .//path/to/whl/ and try again.

If you see a bunch of errors about missing libs you probably forgot to source your venv source /venv/bin/activate

If you get undefined symbol errors (gsm_create, gsm_destroy ect) you compiled using your systems libgsm and not the version in torchaudio. You can work around this by running export LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libgsm.so.1

If everything's working at this point you'll see some warnings about "AMD Radeon RX 580 2048SP with CUDA capability sm_80 is not compatible with the current PyTorch installation." and "Can't initialize NVML" these are both false warnings and can be ignored.

On the first run you should see the script reach out and download a bunch of models

When finsihed you should have a file named "bark_generation.wav"

!codecels