Google DeepMind researchers trained Gemma-4 4B using self-rewarded reinforcement learning to formulate and evaluate arguments drawn from moral dilemmas. After training, the model changed its answer based on the order of the options in 6.6% of cases, down from 14.3%, a reduction of more than half. Its tendency to agree with users’ stated views barely changed, falling from 45.3% to 44.0%, while its accuracy on moral fables declined from 56.5% to 53.4%. The researchers conclude that the training mainly made the model more consistent in its choices without improving its moral judgment or resistance to persuasive users.
Source: Read the original article


