Feature #3164
Updated by Gregory Magarshak 8 months ago
Let's get rid of the streams_metrics table, and instead move the code to Streams_Participant extra named "Metrics/viewed".
And move the code to the Metrics plugin. Replace Streams/metrics/post handler with Metrics/viewed/post handler
The Metrics plugin already has tables like "hit" and "visit", which allow us to record when a user has "hit" a specific URL, during a "visit". And one "visit" contains essentially all hits by a user in a given session. All this already works for URLs on the website, and can be used for analytics. There is also a "canonical URI" in case multiple different URLs are pointing to same URI.
h2. Users and Views. Teasers and Content.
Usually, people will upload and share files/videos/etc. via invite to a stream through our apps (e.g. Groups app). The invited user would have access to see the stream even before they created an account, because Streams plugin grants this readLevel to their session if they came through the right inviteId.
We will probably change this, to allow them readLevel="teaser" until they are logged-in, instead of readLevel="content". So they would be able to watch the teaser, but would have to create an account and log in, if they want to watch the full content.
If user invites by individual email then the invited user automatically has an account. But if user posts an multi-invite link somewhere (i.e. with invitedUserId = ""), e.g. on Twitter or sends email to multiple people, then this invite link will allow up to N people to create account through it, and gain access.
When accepting an invite, the user typically "auto-subscribes" to the stream, thus also joining it and becoming a streams_participant. I would recommend also storing the metrics inside Streams_Participant->setExtra("Metrics/viewed", $metrics). So this way, for example, mousing over avatars in Streams/participant can easily highlight what they watched, or didn't.
h2. Metrics format
The metrics for viewing videos should work as they do now, for logged-in users. We would be able to see the "viewed" metrics for any video, PDF, etc.
The "Metrics/viewed" participant extra should have JSON but it should be moved one level down to be under "intervals" property:
<pre>
{
"combined": [
[1, 2],
[3.5, 7.2]
],
"intervals": [
[1, 2],
[1, 1.5],
[1.2, 1.5],
[3.5, 7.2]
],
"visits": [
visitId, // the ID of visit when interval was watched
visitId,
visitId,
visitId
]
}
</pre>
Note that you are now saving
Have a maximum length in config "Metrics"/"viewed"/"maxLength": 1000. If you run out of room in the JSON, abort saving new "intervals" and "visits", just update "combined".
This would also be used for PDFs, except the intervals would be in % of the file's height. (You store the top and bottom %, as they scroll, debounced by 1000ms.)
h2. Metrics_Viewed table
Then when saving the participants row, we must also have a hook in Metrics plugin to update the generic metrics of the stream. By this, I mean aggregating all intervals into a graph, and storing it in Metrics_Viewed table.
This table should have the following fields:
* publisherId VARBINARY(31) PRIMARY KEY
* streamName VARBINARY(255) PRIMARY KEY
* userMode ENUM ("user", "visit") PRIMARY KEY
* contentMode ENUM ("content", "preview") PRIMARY KEY
* startPosition DECIMAL (10,4) PRIMARY KEY // could be seconds for video/audio or % for PDF
* endPosition DECIMAL (10,4) PRIMARY KEY // could be seconds for video/audio or % for PDF
* viewedCount INTEGER = 0 by default // one per Users userId, or Metrics visitId if logged-out
* insertedTime TIMESTAMP DEFAULT CURRENT_TIME
* updatedTime TIMESTAMP
Notice that this table is not in JSON format. Why? Because if many people are viewing a stream at once, we don't want necessarily to lock the same row with many transactions and update it. Instead, we include "startPosition" and "endPosition" "viewedInterval" in the primary key, so we would lock individual rows corresponding to individual intervals only. It's a little better.
There should be a function @Metrics_Viewed::update($oldMetrics, $newMetrics)@ which should update this table. It would be called from a hook in saving $participant row, if the metrics have changed there. It would use the "combined" values from the participant metrics, only.
Also, you should add in Metrics/config/plugin.json a config under "Streams"/"types"/"*"/"metrics"/quantize: 10, to store a minimum interval length (in seconds), e.g. 10 seconds by default.
Intervals in this table may overlap. For example, if someone watches combined [1, 10] and someone else watches combined [1, 20]. You'll have two different intervals and counts. If someone watched combined [1, 10] and then later watched some more, so now they have combined [1, 20], then in one database transaction, decrement viewedCount on [1,10] and increment on [1,20]. The viewedCount can thus be decremented to 0 on some rows.
For analytics, we will simply SELECT all desired intervals, e.g. where viewedCount > 0, and render them on a Q/chart in a dialog.
h2. Logged-Out users
If a user is not logged in but is able to play the video, the stream type in config may have this config under it:
<pre>
"metrics": { "evenIfNoUserId": false }}
</pre>
In this case, it will call @Metrics_Viewed::update($oldMetrics, $newMetrics)@ even if someone is logged-out but has a valid @Metrics_Visit@ and they watch the video. Use the visitId similar to the "userId" basically, when it comes to incrementing the counts. If a non-user views the same video interval many times in one visit, you still increment only once per visit.
Often, the logged-out users will just be watching teaser versions of videos.