您可以利用面板小程序开发构建出一个基于 Ray 框架的 AI 宠物设备面板。
详见 面板小程序 > 搭建环境。
首先需要创建一个产品,定义产品有哪些功能点,然后再在面板中一一实现这些功能点。
注册登录 涂鸦开发者平台,并在平台创建产品:
面板小程序的开发在 小程序开发者 平台上进行操作,首先请前往 小程序开发者平台 完成平台的注册登录。
详细操作步骤可以参考 面板小程序 > 创建面板小程序。
打开 IDE 创建一个基于 AI 宠物面板模版 的面板小程序项目,需要在 Tuya MiniApp IDE 上进行操作。
详细操作步骤可以参考 面板小程序 > 初始化项目工程。
在项目内配置您在前面 创建产品 > 步骤 10 中复制的智能体 ID,修改模版文件 /src/constant/index.ts
:
// 智能体 ID,修改为产品配置的智能体 ID
const AGENT_ID = "xxx";
使用面板之前要先添加宠物信息,按照引导流程选择宠物类型、品种、性别、活跃度,上传正面照,并填写宠物的昵称、生日、体重等信息。
<View className={styles.content}>
{step === 0 && <PetType value={type} onChange={handlePetTypeChange} />}
{step === 1 && (
<PetBreed
petType={type}
value={breed}
onChange={handlePetBreedChange}
onBack={() => setStep(step => step - 1)}
/>
)}
{step === 2 && (
<PetSex
value={sex}
onChange={handlePetSexChange}
onBack={() => setStep(step => step - 1)}
/>
)}
{step === 3 && (
<PetActiveness
petType={type}
value={activeness}
onChange={handlePetActivenessChange}
onBack={() => setStep(step => step - 1)}
/>
)}
{step === 4 && (
<PetAnalytics goNext={handleGoNext} onBack={() => setStep(step => step - 1)} />
)}
{step === 5 && (
<PetInfo
petType={type}
breed={breed}
sex={sex}
activeness={activeness}
profile={profile}
/>
)}
</View>
const handleSave = async () => {
if (name.trim() === '') {
ToastInstance({
context: this,
message: Strings.getLang('pet_info_name_empty'),
});
return;
}
if (weight === 0) {
ToastInstance({
context: this,
message: Strings.getLang('weight_not_zero'),
});
return;
}
try {
showLoading({
title: '',
mask: true,
});
const petId = await (dispatch as AppDispatch)(
addPet({
petType,
breedCode: breed,
sex,
activeness,
name: name.trim(),
avatar: bizUrlRef.current,
weight: weight * 1000,
birth: birthday,
ownerId: getHomeId(),
timeZone: moment().format('Z'),
tuyaAppId: getTuyaAppId(),
idPhotos: profile?.idPhotos,
features: profile?.features,
})
).unwrap();
dispatch(fetchPetDetail({ petId, forceUpdate: true }));
setNavigationBarBack({ type: 'system' });
navigateBack();
if (store.getState().global.selectedPetId === -1) {
dispatch(setSelectedPetId(petId));
}
} catch (err) {
errorToast(err);
} finally {
hideLoading();
}
};
点击某个宠物可以查看该宠物的详细信息,在详情页面可更新宠物信息、删除宠物记录。
// 更新宠物信息
const handleSave = async () => {
if (name.trim() === '') {
ToastInstance({
context: this,
message: Strings.getLang('pet_info_name_empty'),
});
return;
}
try {
showLoading({
title: '',
mask: true,
});
await (dispatch as AppDispatch)(
updatePet({
id: pet.id,
petType: pet.petType,
breedCode,
sex,
activeness,
name: name.trim(),
avatar: bizUrlRef.current,
weight: weight * 1000,
birth: birthday,
rfid: pet.rfid,
ownerId: getHomeId(),
timeZone: moment().format('Z'),
tuyaAppId: getTuyaAppId(),
idPhotos: profile?.idPhotos,
features: profile?.features,
})
).unwrap();
if (pet.id) {
await (dispatch as AppDispatch)(
fetchPetDetail({ petId: Number(pet.id), forceUpdate: true })
).unwrap();
}
navigateBack();
} catch (err) {
console.log(err);
} finally {
hideLoading();
}
};
// 删除宠物
const handleDelete = async () => {
try {
await DialogInstance.confirm({
context: this,
title: Strings.getLang('tips'),
message: Strings.getLang('pet_delete_tips'),
confirmButtonText: Strings.getLang('confirm'),
cancelButtonText: Strings.getLang('cancel'),
});
showLoading({
title: '',
mask: true,
});
await (dispatch as AppDispatch)(deletePet(pet.id)).unwrap();
navigateBack();
} catch (err) {
console.log(err);
} finally {
hideLoading();
}
};
将宠物图片上传到云端进行分析,识别宠物的体型、毛色、表情等外观特征,从而在宠物进食时能准确识别具体的宠物。
首先,请求云端获取文件上传签名及对应的 objectKey
。然后,将图片上传至指定服务器,完成后将 objectKey
作为参数传递给特征分析接口。该接口会返回一个 taskId
,之后通过轮询特征分析结果接口获取分析进度。若返回的 analysisResult
值为 2
,则表示分析成功;若值为 1
,则表示分析失败。根据返回的结果展示相应的分析信息。
const handleChooseImg = async () => {
let paths = [];
try {
paths = await chooseImage(3);
} catch (err) {
return;
}
// 先进入下一阶段
enter('analyzing');
setState({
analyzingText: Strings.getLang('add_pet_analytics_upload_img'),
});
const controller = new AbortController();
controllerRef.current = controller;
let images: Array<{ objectKey: string }> = [];
try {
images = (await Promise.all(paths.map(p => uploadImage(p, 'petFeature')))).map(d => ({
objectKey: d.cloudKey,
}));
if (controller.signal.aborted) {
return;
}
} catch (error) {
enter('failed');
return;
}
let idx = 0;
const tips = [
Strings.getLang('add_pet_analytics_upload_tip1'),
Strings.getLang('add_pet_analytics_upload_tip2'),
Strings.getLang('add_pet_analytics_upload_tip3'),
Strings.getLang('add_pet_analytics_upload_tip4'),
Strings.getLang('add_pet_analytics_upload_tip5'),
];
const id = setInterval(() => {
setState({
analyzingText: tips[idx++ % tips.length],
});
}, 3000);
try {
const taskId = await aiPetFeature({
ownerId: homeId,
images,
miniAppId,
agentId: AGENT_ID,
});
const [infoRes, similarRes] = await Promise.all([
loopGetAnalysisResult({
taskId,
controller,
type: AnalysType.Profile,
}),
loopGetAnalysisResult({
taskId,
controller,
type: AnalysType.MatchPet,
}),
]);
if (similarRes?.matchedPets?.length) {
setState({
similarShow: true,
petResInfo: infoRes,
});
} else if (infoRes) {
emitter.emit('selectProfile', infoRes);
enter('success');
} else {
// 超时
enter('failed');
}
} catch (error) {
enter('failed');
return;
} finally {
clearInterval(id);
}
};
// 上传图片
export async function uploadImage(filePath: string, bizType: UploadFileBizType) {
const fileName = parseFileName(filePath);
const signInfo = await fetchUploadSign({ bizType, fileName });
const manager = await getFileSystemManager();
const data = manager.readFileSync({
filePath,
encoding: 'base64',
});
const { url, objectKey } = signInfo;
await customFileUpload({ url, file: data?.data });
return { cloudKey: objectKey };
}
主要展示家庭下宠物进食情况,当宠物来进食时,会识别出具体哪只宠物,并生成一条进食记录。
const [eatingRecords, setEatingRecords] = useState<IEatingRecord[]>([]);
const getData = async (pageNo: number, isFresh?: boolean) => {
try {
const day = dayjs();
const startOfDay = day.clone().startOf('day').valueOf();
const endOfDay = day.clone().endOf('day').valueOf();
setRefreshing(!!isFresh);
const eatParams = {
ownerId: homeId,
uuid: getDevInfo().uuid,
startTime: startOfDay,
endTime: endOfDay,
pageNo,
pageSize: 100,
};
const { pageNo: pageNumber, hasNext, data = [] } = await fetchPetEatingRecordApi(eatParams);
setRefreshing(false);
setHasNext(hasNext);
setCurrentPageNo(pageNumber);
const formattedEatingRecord = data.map(item => {
const petName = Array.isArray(item.pets)
? item.pets.map(p => {
return find(pets, { id: p.petId })?.name;
})
: Strings.getLang('pet');
return {
...item,
timeStamp: item.recordTime,
type: RECORD_DATA_TYPE.feed,
desc: Strings.formatValue('dsc_feed_eating', petName),
};
});
if (pageNo > 1) {
setEatingRecords([...eatingRecords, ...formattedEatingRecord]);
} else {
setEatingRecords(formattedEatingRecord);
}
} catch (error) {
setRefreshing(false);
console.log('fetch error: ', error);
}
};