The frame-indifferent thermodynamically-consistent model of thermoviscoelasticity at large strain is formulated in the reference configuration with using the concept of the second-grade nonsimple materials. We focus on physically correct viscous stresses that are frame indifferent under time-dependent rotations. Also elastic stresses are frame indifferent under rotations and respect positivity of the determinant of the deformation gradient. The heat transfer is governed by the Fourier law in the actual deformed configuration, which leads to a nontrivial description when pulled back into the reference configuration. Existence of weak solutions in the quasistatic setting, i.e. inertial forces are ignored, is shown by time discretization.