DolceSDK - Playstation Vita homebrew SDK

[cuevavirus]

Development of DolceSDK has ended to focus on Vita Development Suite.

[cuevavirus]

Changes summary 2020-04-06

toolchain

- CMake macros have fixed dependency management and will no longer have circular dependency error.
- Changes to dolce_create_stubs to allow easy creation of static libraries from stubs
- Added dolce_gen_libs, a new cmake function that allows easy creation of static libraries from yml NID databases

headers

Additions and fixes to  SceJPEG, SceAvPlayer, SceRegMgr, ScePower, SceThreadmgr/Processmgr, SceAudio, SceSblACMgr, SceSysmem, SceLibKernel, SceDisplay

packages

Added ffmpeg and lame (https://github.com/DolceSDK/packages/commit/7e4452a49231d94a7ab18560bc1a28a83b7dddd5)

samples

GXM samples restored (https://github.com/DolceSDK/samples/commit/269651fed1935707023a2a916a8f182dd852ef18)

buildscripts

gdb removed due to intermittent build errors (https://github.com/vitasdk/buildscripts/issues/65)

[cuevavirus]

Changes summary 2020-04-19

toolchain

Thanks to Graphene, the toolchain is able to create apps never before seen with Vitasdk.

- fixed a bug in velf generation that prevented most common dialogs from being used
- added dolce-mkbootparam, a new command for creating custom boot params
- changes to dolce-make-fself and dolce_make_self to allow using custom boot params made with the previous command

Dolcesdk/vitasdk makes use of mksfoex, originally from pspsdk. We have included the pspsdk license which vitasdk had failed to do.

headers

Changes to SceIftu.

[cuevavirus]

Changes summary 2020-04-21

toolchain

- dolce-mkbootparam renamed to dolce-make-bootparam and updated with all boot param options. See sample usage at https://github.com/cuevavirus/BG-App-PSV

headers

- added psp2/systemgesture.h for SceSystemGesture

packages

- added libvita2d_sys

[cuevavirus]

Changes summary 2020-05-18

packages

- packages are now hosted separately from Vitasdk
- added binaries for Dolcesdk exclusive packages: lame, ffmpeg, libvita2d_sys

ddpm

- updated to download packages from new package host

headers

- names and prototypes in SceSysclibForDriver, SceGxmInternalForVsh, SceLibKernel

[cuevavirus]

Changes summary 2020-06-06

Compatibility notice: The gcc code gen options "-fno-short-enums" and "-fshort-wchar" are now default as of build-2020-06-05-225312. This fixes enums to 4 bytes and changes the type of wchar to unsigned short. This conforms with the SNC toolchain and provides binary compatibility with the SNC toolchain. Also fixes some wrong struct size in the headers caused by variable size enums. Linking against older binaries may produce a linker warning and can be ignored if you do not use enums and wchars, or have used them in a way that is unaffected by this change. Packages in the DolceSDK package repository will be rebuilt over the coming days.

Deprecation notice: The header files "dolcesdk.h" and "dolcesdkkern.h" will be removed in the future. The number of headers in DolceSDK is increasing and including too many headers can negatively impact compilation speed.

headers

- SceNotificationUtils thanks to Graphene
- Additions for SceKernelThreadmgr, SceGxm, SceGxt, SceScreenShot, SceIme, SceImeDialog

packages

- added fnblit
- updated taihen

[cuevavirus]

Changes summary 2020-06-17

As of build-2020-06-17-235400, SceLibc is useable including memory allocation functions. Newlib remains the default but consider this a transitional period while we work towards removing newlib entirely. Follow these steps to use SceLibc:

1. Pass "-nostdlib" to gcc to remove default library links.
2. Link SceLibc_stub, as well as any other necessary libraries.
3. Define _start() instead of main(). These two functions do not share the same arguments.
4. Pass "-h <bytes>" to vita-create-elf specify the libc heap size. The default is 256 KiB if you do not pass this option. Set DOLCE_ELF_CREATE_FLAGS if you are using the CMake macros.
5. Pass "-fno-builtins" if you do not want gcc to use libc builtins.

This has been made possible with help from Graphene.

newlib

- Remove "-fshort-enums"

gcc

- Use "int" and "unsigned int" for "int32_t" and "uint32_t" instead of "long int" and "long unsigned int"

packages

- Various packages added and rebuilt for Dolcesdk

headers

- Added or updated headers for SceGxm, SceNotificationUtil, SceIncomingDialog, SceShellUtil, SceSas

[cuevavirus]

Changes summary 2020-07-07

Compatibility notice: In build-2020-07-07-222810, a large number of header changes occurred relating to SCE types and threadmgr. You may need to include additional headers, rename struct fields, rename union fields, rename macros, add field for typedefs and structs that became unions, or remove duplicate definitions. Threadmgr related structs which had the wrong number of fields had also been corrected. With these changes, psp2/kernel/threadmgr.h has all constants and prototypes for thread management and synchronisation.

Headers

- Additions and changes in SceBgAppUtil, SceTriggerUtil, SceThreadmgr, SCE types, error codes, SceCodecEngine, SceJpegArm, SceModulemgr (many of these additions are thanks to Graphene)
- Items common to user and kernel modules are to be placed in psp2common. Do not include directly from psp2common.
- Added a file db_367.yml containing NIDs for firmware >=3.65. At the moment this file is for reference only.

ddpm

- Bug fix related to quoting variable expansion
- Updated default install list to match available packages

makepkg

- Bug fix related with dependency management (or lack thereof)

toolchain

- Made LIB argument for dolce_gen_libs and dolce_create_stubs optional

packages

- Removed various unused packages
- Added libvitaSAS and libShellAudio
- Patched most packages to build with Dolcesdk

[cuevavirus]

Changes summary 2020-08-23

It's recommended to rebuild all dependencies due to changes to compiler and ABI.

headers

- 3.65 stubs added
- fixed async functions for SceIofilemgr, add SceSas internal functions (credits: Graphene)
- added NIDs or prototypes for SceDbg, SceShellSvc, SceNgs, SceIofilemgr, SceLibKernel, SceThreadmgr, SceAppMgr, SceAVConfig, SceSysclibForDriver, SceGxm, SceFiber, SceUlt, SceLibSsp, SceRtabi
- fixed incorrect NIDs
- merged changes from vitasdk

With the completion of SceSysclibForDriver and SceRtabi, in most cases you will no longer need to link libgcc. libk has been removed from packages as well.

ABI

- char is signed by default

Be aware that this change can change comparison semantics. For example, a literal 255 has type int and in a comparison with a signed char, the signed char is converted to a int, and the comparison will never hold true. Enable gcc -Wall and -Wextra to check for such errors.

toolchain

- updated gcc to 9.3.0, binutils to 2.35, libzip to 1.7.3
- add __VITA__ as builtin define
- adjust gcc default cpu tuning options
- bug fixes and enhancements for mksfoex (credits: Graphene)
- bug fixes and enhancements for libs-gen, cmake functions, and cmake toolchain file

ddpm

- removed wget usage
- bug fixes

newlib

- set environmental variable HOME in crt0
- bug fixes

[cuevavirus]

Changes summary 2020-08-30

There has been several breaking changes. Please read this note in detail.

ddpm

- The installation and sdk update procedure has changed. Please see https://github.com/DolceSDK/doc.

headers

- All sce_paf_private_xxx functions have been rename sce_paf_xxx.
- Added SceGxmInternal functions (credits: Graphene).
- Added SceFios2 headers.

toolchain

- dolce-pack-vpk has been removed and replaced with dolce-make-pkg. No changes have been made to the CMake function dolce_create_vpk, but the variable DOLCE_PACK_VPK_FLAGS has been removed. Instead use DOLCE_MAKE_PKG_FLAGS.
- right.suprx will be automatically included by dolce-make-pkg if requirements are fulfilled (see https://github.com/DolceSDK/customright).
- Livearea assets will be added automatically if not explicitly added. This fixes a problem where apps cannot be opened due to lacking livearea assets.

buildscripts

- The Windows cross build has been made about 25% faster. On travis-ci this resulted in about a 10 minute reduction in build time.

[cuevavirus]

Changes summary 2020-09-09

headers

- added functions in SceThreadMgr and SceLibKernel (credits: Graphene)
- added functions in ScePaf, SceRegMgr

toolchain

A major bug caused by linker misconfiguration has been patched. The bug caused a per module waste of 32 KiB of runtime memory, and between 32 and 64 KiB of ELF file size. It is recommended to rebuild all SELFs.

Here is a comparison of section and segment sizes before and after the patch https://gist.github.com/cuevavirus/e9863502adfccfc18216b3883fa72a45. Note that the total section size in file went from 74933 to 21685 bytes which saves 53248 bytes. More significantly, the runtime memory size of the RX segment went from 37253 to 4485 which saves exactly 32 KiB. As Princess of Sleeping said, this waste of memory is FATAL.



Princess of Sleeping had previous reported this problem to the Hankaku discord server.



Vitasdk git commit pusher Francisco José García García directed us to this issue from 2016, which the only activity thereafter is Francisco José García García tagging it as "enhancement".



Two things are clear. First, we have shown here that contrary to Yifan Lu's claim, the side effects are not mitigated with SELF compression. Second, Vitasdk is content with merely "it works" and does not further consider "how can we make it better". This is a significant toolchain issue and after 4 years, much less fixing it, they do not even understand its consequences.

The DolceSDK team is committed to bringing you the best performing and most cutting edge PlayStation Vita homebrew SDK.

[cuevavirus]

Changes summary 2020-09-23

Due to changes in db.yml and stub generation, it is likely that user modules will require adjustments to linked libraries to build. Please read below.

packages

- added box2d (credits: coderx)

toolchain

- binutils reverted to 2.32 due to issues in 2.35
- discard debug sections from ELF
- bug fix in dolce-make-fself (credits: princess of sleeping)
- dolce-libs-gen will now always generate one stub library file per library, whereas before that was only the case for kernel libraries, and all user libraries of a module are collected into one stub library

headers

Please note that SceGxmInternal functions have been renamed, and previously deprecated headers have been removed.

- added NIDs and prototypes in SceIniFileProcessor, SceKernelForMono (credits: graphene)
- fixes in SceCpu, added protypes in SceFios2Kernel (credits: princess of sleeping)
- added NIDs and prototypes in SceLibXml, SceGxm
- removed deprecated headers
- removed functions non-existent in 3.60 retail firmware
- moved libraries to correct module, and functions to correct library

The last point of moving libraries to the correct modules have been repeatedly brought up to Vitasdk maintainers. In fact you can see in 477 that it was brought up yet again while I was working on these very changes.

https://github.com/vitasdk/vita-headers/pull/410
https://github.com/vitasdk/vita-headers/issues/439
https://github.com/vitasdk/vita-headers/pull/477







For some reason these changes have made this person irrationally angry. DolceSDK has complied with Vitasdk's and Pspsdk's license in all applicable works.



The reason for refusal is yet again "backward compatibility". This reason is only a convenient excuse for the maintainers. When making their own changes, they do not hesitate to think about backward compatibility.

The most recent example being the upgrade to gcc10 (https://github.com/vitasdk/buildscripts/commit/9655cf26b2fd182108bf877a7db9d215ed32b97e), which is widely known for causing breakages due to -fno-common becoming the default (https://gcc.gnu.org/gcc-10/porting_to.html). Some of the most popular homebrews such as Vitashell (https://github.com/TheOfficialFloW/VitaShell/issues/579) are broken.

Another example is this inexplicable change to gcc configuration (https://github.com/vitasdk/buildscripts/commit/ba44b032963937730208151054ba5c673be6fc75). It was basically unuseable.



Still, it took more than a month before this change was reverted (https://github.com/vitasdk/buildscripts/commit/3e857643e84f57eb142c62efb2de98749138a232).

The DolceSDK team is focuses on correctness, consistency, and good design. Backward compatibility is neither a primary concern, nor an excuse for refusing changes.

[cuevavirus]

Changes summary 2020-10-03

toolchain

make-fself has been completely rewritten and is now capable of producing the exact same output as make_fself.exe, except for revised section headers, which is not implemented. Some options of make_fself.exe are also not implemented. The commands produce the same output.

Code: [Select]

# without compression
dolce-make-fself in.elf out.self
make_fself.exe -r -e in.elf out.self

# with compression
dolce-make-fself -c in.elf out.self
make_fself.exe -r -e -c in.elf out.self

A new option `--strip` has also been added. This option will strip the ELF file except for the ELF header, the program header table, and segments referenced by the program headers except for any of type PT_SCE_COMMENT. This option produces the same output as using psp2pubcmd.exe, with the exception of the program version and extra zero bytes at the end of the certified file added by psp2pubcmd.exe. `--strip` is always enabled when using the dolce_create_self CMake function.

You will no longer see decompression error in 🟊Check.

headers

- additions and fixes to SceGim, ScePsp2Compat, SceAppMgr, SceIniFileProcessor, SceGpuEs4, SceIofilemgr (many changes are thanks to Graphene)

[cuevavirus]

Changes summary 2021-01-05

Contributors: Graphene, SonicMastr, Isage, striker.sh

library

- A number of prx and static libraries are now bundled with the SDK installation. Includes libScePiglet, libshacccg, libSceScream, among others.
- Added prototypes or NIDs for SceAudiocodec, SceAvdec, SceVideodec, SceShaccCg, SceMotion, ScePaf, ScePsp2Compat, ScePiglet, SceScream, SceSndstream, network related modules, NP related modules
- Header additions and fixes for SceAppmgr, SceIofilemgr, SceIniFileProcessor, SceSysmem, SceDisplay, SceNotificationUtil, SceGxm, SceAudiodec, SceAudioenc, ScePromoterUtility, SceAppUtil, common dialog related headers, SceNetAdhoc, SceJson

packages

- New libraries libpib, glfw.
- SDL and SDL2 ports have been rewritten with significant improvements. SDL2 renders with GLES2 using libpib.
- Build prx libraries for vita2d_sys and vitaSAS
- Updated version for packages

sample

- New samples have been added for libshacccg, GLES2 with libScePiglet, notifications from BG app, SceIniFileProcessor

ddpm

- Scripts rewritten for improved usability and ease of maintenance

gcc

- Add builtin define __psp2__

newlib, pthread, toolchain

- Bug fixes and enhancements