So reading the man page for alerts does the following make sense. Configure a rule for with no duration for the primary oncall contact and then create a second rule matching the first, but include DURATION>15. If the above makes sense, I am assuming that if the primary oncall contact acks the alert within the 15 minute span, the second alert would not be triggered.